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Preface 


It is very tempting but a little bit dangerous to compare the style of two 
great mathematicians or of their schools. J think that it would be better to 
compare papers from both schools dedicated to one area, geometry and to 
leave conclusions to a reader of this volume. 


The collaboration of these two schools is not new. One of the best 
mathematics journals Functional Analysis and its Applications had I.M. 
Gelfand as its chief editor and V.I. Arnold as vice-chief editor. Appearances 
in one issue of the journal presenting remarkable papers from seminars of 
Arnold and Gelfand always left a strong impact on all of mathematics. 


We hope that this volume will have a similar impact. Papers from 
Arnold’s seminar are devoted to three important directions developed by 
his school: Symplectic Geometry (F. Lalonde and D. McDuff), Theory of 
Singularities and its applications (F. Aicardi, I. Bogaevski, M. Kazarian), 
Geometry of Curves and Manifolds (S. Anisov, V. Chekanov, L. Guieu, E. 
Mourre and V. Ovsienko, S. Gusein-Zade and S. Natanzon). A little bit 
outside of these areas is a very interesting paper by M. Karoubi Produit 
cyclique d’espaces et opérations de Steenrod. 


Papers from Gelfand’s seminar are complimentary to a recent Gelfand 
seminar volume and are more or less related by the notion of integral trans- 
forms. An application of Radon transforms to a solution of Hilbert’s fourth 
problem is given by J.C. Alvarez, I. Gelfand and M. Smirnov, nonlinear 
integrable equations and nonlinear Fourier transform are considered by A. 
Fokas, I. Gelfand and M. Zyzkin. Two papers (I. Gelfand, M. Graev and A. 
Postnikov, A. Kazarnovski-Krol) are devoted to hypergeometric functions 
— a cornerstone of modern theory of such functions developed by Gelfand’s 
school — is also a Radon transform of homogenous functions. 
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Discriminants and local invariants 
of planar fronts 


Francesca Atcardi 


Introduction 


The aim of this study is the description of all the local additive invari- 
ants of the plane wave fronts. A generic wave front is a curve whose 
only singularities are the transversal self-intersections on the semicubi- 
cal cusps (see fig. la,b). The invariants that we shall find are “dual” 
to different strata of the discriminant formed by the nongeneric wave 
fronts. 

For instance, our results imply the following minoration of the num- 
bers of different events of nongenericity on any path connecting the two 
curves w3,9 and 3.9 shown in fig. 1b: 

i) the number of self-tangencies with parallel coorientations is at least 
2; 

ii) the number of self-tangencies with antiparallel coorientations is at 
least one; 

iii) the number of cusp crossings is at least 4, provided that the triple 
points are avoided; 

iv) the number of triple points is at least 2, provided that the cusp 
crossings are avoided. 


Q K3,0 W3 
(a) (b) 


Figure 1. Oriented and cooriented planar fronts 


The proofs are based on the introduction of some topological in- 
variants of the generic wave fronts, whose jumps at the crossing of the 
corresponding nongeneric events are fixed. The calculation of the val- 
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ues of these invariants on the curves w39 and K’39 provides the above 
minoration. 

There are five basic invariants (independent of the orientations of 
the front and plane as well as of the coorientation of the front: n 
(the number of double points), \ (the number of cusp points), J* 
(responsible for the parallel coorientations tangencies), J~ (responsible 
for the antiparallel coorientations tangencies), Sp (responsible of the 
triple points and cusp crossing). 

The invariants J*, J~ and Sp of wave fronts reduce to the invari- 
ants J+, J~ and St for those wave fronts which have no cusps, i.e., to 
the Arnold invariants of smooth closed plane curves. 

The number of the independent local additive invariants of generic 
oriented and cooriented wave fronts on the oriented plane (counting 
both those invariants, like J+, which are independent of the orienta- 
tions and those which depend on the orientations of the front and plane 
as well as of the coorientation of the front) is equal to 10. One of these 
invariants is the Maslov index p. We introduce 6 new basic invariants 
Cae oa Xr, a pl, p*). 

The invariant f* (f~) is counting the double points where two pos- 
itive (negative) branches of the front do intersect (the branch is positive 
if its orientation and coorientation define the positive orientation of the 
plane). 

The invariant A! (A!) is counting the number of cusps at which 
the orientation of the plane defined by the two branches of the oriented 
curve leaving the cusp point (first along the coming branch, second 
along the outgoing branch) is positive (negative). 

The invariants p!', p! are responsible for the cusp crossings taking 
some sign rule into account (see §3). These six new invariants verify 
the relations: 


1 
Ee Sy, et I ae a 


hence only 4 of them are really new. 

A planar front is the projection to R? (with coordinates «x, y) of a 
legendrian curve. A legendrian curve is the image of a C!-immersion 
of S? in the space M° (with coordinates z, y, @(mod27) for cooriented 
fronts, (modm) for noncooriented fronts) of the contact elements of the 
plane, with its natural contact structure 


(cosd)dz + (sing)dy = 0. 


We call such an immersion of $1 into M? an L-immersion. 
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The front is cooriented, if the contact element is cooriented, i.e., 
if a choice of one of two half-planes into which it divides the tangent 
plane is made. We shall now consider the cooriented fronts, and M* 
will be the space of the cooriented contact elements. 

The front is oriented if the preimage circle S* is oriented. 

A front class is a class of L-immersions having generic fronts up to 
diffeomorphisms of the plane and preimage circle. 

In the space of L-immersions the immersions having nongeneric 
fronts form a hypersurface (a subvariety of codimension one) called the 
discriminant. 

A coorientation on a smooth hypersurface in a functional space is 
a choice of one of the two parts separated by this hypersurface in a 
neighborhood of any of its points. This part is called positive. The 
discriminant (or a connected component of it) is called coorientable if 
the coorientations of its parts separated by strata of codimension 1 on 
it (of codimension 2 in the ambient space) are consistent. This means 
that the intersection index of any generic small oriented closed curve 
with the cooriented hypersurface should vanish. 

In the functional space the connected components of the comple- 
ment to the discriminant are the front classes. A local invariant of a 
front class is a function constant over each of such components. Local 
means that its increment under a generic crossing of the discriminant 
depends only on the behavior of the family of immersions in the neigh- 
borhoods of the points of the preimage circle sent to the points involved 
in the singularity. 

A local invariant is thus defined up to an additive constant in terms 
of its jump at the crossing of the discriminant, provided that such 
discriminant is coorientable and that it defines in the ambient space a 
trivial 1-cocycle. 

In fact every cooriented stratum of codimension 1, provided with a 
jump at its crossing in the positive direction (i.e., from the negative to 
the positive side defined by the coorientation), gives a 1-cocycle. A 1- 
cocycle is a function defined on generic curves in the ambient space: the 
value of a 1-cocycle on a generic curve is defined by the sum of the jumps 
(with signs) on all the crossings of the curve with the corresponding 
stratum. 

The study using this approach of the immersion of a circle in the 
plane [2] leads to the definition of three basic invariants. One proves 
that the space of immersions with a given index is connected. In this 
space the discriminant has three different components, all coorientable, 
corresponding to three different types of degenerations (see fig.2): 

1) self-tangency point with parallel orientations, 
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2) self-tangency point with antiparallel orientations, 
3) triple point. 


Figure 2. Parallel self-tangency, antiparallel self-tangency 
and triple point 


The choice of the jumps at these strata thus defines three 1- 
cocycles. These cocyles are trivial. The corresponding invariants are 
called J+, J~ and St; the jumps are chosen so that J+ — J~ gives 
the number of double points of the immersions while the jump at the 
stratum of the triple points is 1 by a rule explained below. 

The additive constants are chosen so that the three invariants result 
in being additive under the connected sum of immersions: it is sufficient 
to require that the values of the three basic invariants be all equal to 
zero on the class of the circle. 

The values of the three invariants are thus well defined on the 
basic curves of every index. The values of the invariants of a generic 
immersion a of a given index are thus defined by whatever path in the 
space of immersions connecting the basic curve with a. Their increment 
along this path is in fact proportional to the intersection index of the 
path with the corresponding stratum. It follows that the difference of 
the values of a local invariant on two different immersions minorates 
the number of singular events of the corresponding type that occur in 
the course of a generic perestroika transforming one into the other. 

Any linear combination of the basic invariants is again an invariant; 
some of them are interesting, for example the number of double points 
(J+ —J~) and € = Jt + 2St which vanishes on the tree-like curves [2]. 

Coming back to the fronts, a smooth plane closed curve, provided 
with a normal vector at every point, i.e., with an unitary vector or- 
thogonal to the tangent vector and forming with it a frame positively 
orienting the plane, is the front of a legendrian curve which is nowhere 
vertical (nowhere tangent to the fibre of the fibration M3 —> Te), 
Thus the classification of the fronts must contain the classification of 
the immersions of a circle in the plane. 

The space of L-immersions is not so simple as the space of the 
immersions of a circle in the plane, and the study of its topology is 
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preliminary to the definition of the local invariants. 

To every oriented legendrian curve in the space of cooriented con- 
tact elements of the plane one associates two integers: the index i and 
the Maslov index js (see [1]). Both these indices can be calculated in 
terms of the front of the legendrian curve. 

The index 7 of an oriented legendrian curve in the space M° is equal 
to the total angle (divided by 27) of the rotation of the coorienting 
normal vector of its front when a point on the front makes a full turn 
along the curve. 

The Maslov index yz of a generic oriented legendrian curve is equal 
to the angle (divided by 7) of the rotation of the tangent vector to the 
curve in the contact plane during a complete turn of the preimage point 
along the preimage circle. In terms of the front, the Maslov index is 
equal to the difference between the number jz* of positive cusps and 
the number jz~ of negative cusps: 4 = wt —y~. (A cusp of an oriented 
and cooriented front is called positive if the coorienting normal vector 
at the cusp point belongs to the half-plane bounded by the tangent line 
at the cusp and containing the cusp branch with the orientation going 
away from the cusp point). 

The sign of the index does not depend on the front coorientation 
and changes when the orientations of the plane or front are changed. 
The Maslov index changes its sign when the orientation or the coori- 
entation of the front is changed and is independent of the orientation 
of the plane. 

The space of all L-immersions with given index 7 and given Maslov 
index ps is denoted by ();,,. 

One proves that (2;,,, is connected but not simply connected [3]. 

An L-immersion is called marked if a given (once and for all) point 
of the preimage circle is sent to a given (once and for all) cooriented 
contact element of the oriented Euclidean plane. 

The group G3 of Euclidean motions of the plane acts without fixed 
points on the space of L-immersions, which is thus homeomorphic to 
the direct product of the space of the marked L-immersions 0? , with 
G3 BS) x R?. 

The discriminants and their coorientations are G3-invariant. Hence 
in the construction of the invariants one can consider only the marked 
immersions. The space OQ? ms (weakly) homotopy equivalent to a circle, 
namely the circle of all possible directions of the marked legendrian 
curve in the contact plane at the chosen point. 

It follows that a hypersurface of the discriminant stratum of codi- 
mension 1 in Q? , defines a corresponding invariant if it satisfies the 
following. two conditions: 
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1) it is cooriented, 

IT) the cooriented hypersurface is cohomologous to zero in oO? y (its 
intersection indices with all the closed loops vanish). 

The study of the discriminant of the self-tangencies, called stratum 
kK (from the Russian word kasanie meaning tangency), has been done 
by V. Arnold [1]. The two components Kt and K~ corresponding to 
the tangencies with parallel and antiparallel normal vectors are coori- 
entable. Moreover, it is proved that the l-cocycle defined by the jump 
at the cooriented stratum K* is cohomologous to zero. Then the values 
of the corresponding invariant J+ (which is a natural generalization of 
the invariant J* defined for the immersions of a circle in the plane) 
are chosen on the basic curves belonging to the spaces 2;,,,, so that the 
resulting invariant is additive under the connected summation of two 
oriented and cooriented fronts. 

The invariant J*, unlike the index and the Maslov index, is inde- 
pendent of the orientation and of the coorientation of the front and of 
the orientation of the plane. 

This work answers to the questions: 

What are all the possible local additive invariants of fronts, inde- 
pendent of all the orientations and coorientation? 

Are there local additive invariants of the fronts which depend on 
their orientation or on their coorientation or on the orientation of the 
plane? 

To answer these questions we proceed in the following way: 

i) We classify all the codimension 1 strata of the discriminant with 
and without the condition of independence of the orientations and the 
coorientation. We provide every such strata with a local coorientation 
and we define the value of the elementary jump of a corresponding local 
function, which is nonzero only at the crossing of that elementary local 
stratum. 

ii) We impose the coorientability condition at every stratum of 
codimension 2: the increment of the local invariants along a small close 
path linked with this stratum must vanish. So we obtain a system 
of linear equations (coherence system) for the jumps defining the 1- 
cocycles. 

iii) We choose a simple set of linearly independent generators of the 
space of solutions of the coherence system. The corresponding linear 
combinations of the elementary jumps define the basic 1-cocycles. 

iv) We verify the vanishing of cohomology classes of the l-cocycles. 
The jump of every basic 1-cocycles is thus the jump of a basic invariant. 

v) We define the values of the basic invariants on the basic curve 
in Q;,, for any i and yp, so that the invariants are additive under the 
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connected summation of two fronts. We prove that these invariants are 
unambiguously defined by the additivity condition. 
The results are the following: 


Theorem 1. Every additive local invariant of the fronts that does not 
depend on their orientation, their coorientation and on the orientation 
of the plane is a linear combination of five linearly independent basic 
invariants. 


Theorem 2. Every local additive invariant of the fronts (that may 
depend on orientation, coorientation or on the orientation of the plane) 
is a linear combination of 10 (including the Maslov index) linearly in- 
dependent basic invariants. 


Remark. The investigation of the local basic invariants starting 
from the local strata of the discriminant dependent on the orientations 
leads to the definition of all the invariants, which are dependent on 
the orientations in different ways. However, I divide the problem into 
two parts, finding first the basic local invariants independent of the 
orientations. This seems to me more useful. 


1. The codimension 1 local strata of the discriminant 


1.1. Classification of the codimension 1 local strata of the 
discriminant 

In the space of L-immersions, the discriminant, consisting of leg- 
endrian curves having nongeneric fronts, has four components corre- 
sponding to different singular events: self-tangencies, triple points, cusp 
crossing and cusps births. They are denoted ilipy thevletters 1<, /7all 
and A respectively (II for the russian prokhozhdenie meaning passing 
and A from the Russian lastochki khvost meaning swallow tail). 

One can, however, on every component locally distinguish different 
events, depending on the different possibilities of the orientation of the 
plane and of the orientation and coorientation of the segments of the 
curve containing the point involved in the singularity. 

Consider the three following involutions acting on the space of L- 
immersions: 

the involution a which reverses the orientation of the plane (aa); 

the involution 6 which reverses the orientation of the preimage cir- 
cle, so reversing the orientation of the front; 

the involution c which reverses the coorientation of the contact 
elements, so reversing the coorientation of the front. 

These commuting involutions generate a group, called RO (for ori- 
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entation reversing)|4]. All other elements of the group different from 
the identity (having still order two) are ab, ac, bc, abc. 

Given a particular nongeneric event occurring on a front, the action 
of RO transforms it into different events. The number of distinguish- 
able events among them (up to rotations of the plane) depends on the 
symmetry type of the event. These different events are identified in the 
classification which is invariant under all involutions, and are distin- 
guished in the classification which takes the orientations into account. 
In the following tables both classifications are shown. The subgroup 
Riz)(y) of RO indicates the symmetry type of the event, i.e., the sub- 
group generated by the involutions x and y with respect to which the 
event is invariant. There are no singular local events invariant for the 
entire group RO. We shall call RO-independent a class of local strata 
which is invariant under the actions of the group RO. Moreover, we 
shall say “orientations” for short instead of “orientation of the plane, 
orientation and coorientation of the front.” 


Lemma 1.1. The discriminant containing L-immersions having non- 
generic fronts with a self-tangency contains 10 different local strata 
(ei spa) dependent on the orientations, corresponding to 4 
RO-independent local strata (KG,..., Kp), shown in table 1. 


The discriminant containing L-immersions having nongeneric 
fronts with a triple point contains 24 different local strata 
(aaa ...,Tg _) dependent on the orientations, corresponding to 5 
RO-independent local strata (T4,...,T}!), shown in table 2. 

The discriminant containing L-immersions having nongeneric 
fronts with a cusp crossing contains 16 different local strata 
(ile; sae Ip;) dependent on the orientations, corresponding to 2 RO- 
independent local strata (IIc, Up), shown in table 8. 

The discriminant containing L-immersions having nongeneric 
fronts with a cusps birth contains 4 different local strata al oN ) 
dependent on the orientations, corresponding to 1 RO-independent local 
stratum (A), shown in table 4. 


Proof. We start with the explanation of the notations used to specify 
the local strata. 


Definition 1.1. A smooth segment of an oriented and cooriented 
front is called positive (negative) if the pair (orienting tangent vector, 
coorienting normal vector) orients the plane positively (negatively) 


. 


Definition 1.2. A cusp point of an oriented front is called of type up 
or down if looking at the segment containing it so that the orientation 
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F(ab)(be) 


Table 1. Local strata of the discriminant K RO-independent, 
RO-dependent and their symmetries. 


goes from left to right, the cusp is pointing upward or downward. (The 
plane is here supposed to be oriented by the frame (to the right, up)). 


Remark 1.1. The character up or down is coorientation independent. 


Remark 1.2. A cusp point is positive either if it is of type up and 
the branch going away from it is positive or if it is of type down, the 
outgoing branch being negative. 


The stratum K of self-tangencies. The upper symbols + and —, 
chosen in [1], mean respectively the parallelism and the antiparallelism 
at the tangency point of the coorienting normal vectors of the two 
tangent segments. The lower symbol C' or D means that the orienting 
vectors at the tangency point are parallel (concordant) or antiparallel 
(discordant). The lower symbol + or — is the sign of both tangent 
segments, when they coincide. 
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For the other symbols consider the area bounded by the two seg- 
ments crossing each other in two points after the tangency event. The 
lower symbol 7 (internal) or e (external) means that the coorienting 
vectors on both segments lie inside or outside this area. The lower 
symbol i+ or i~ means that the only normal vector lying inside this 
area belongs to a positive or negative segment. (equivalent notations 
for it and i~ should be e~ and et, respectively). 

The involutions a and b (whose action on the local strata is shown 
in table 1 by a double arrow) reverses the lower indices (+,—) of the 
strata K (including the signs of i and i~). The involution c (whose 
action on the strata is shown in table 1 by a dotted arrow) reverses 
the same signs and interchanges i with e, so that 77 and 7~ remain 
unchanged. Only the characters C and D and the upper signs meaning 
“parallel” and “antiparallel coorientations” are RO-independent. 


The stratum T of triple points. Consider the three segments meet- 
ing at the triple point as the diagonals of a regular hexagon. The triple 
point is of type B or A if the union of the three oriented segments has 
or does not have the symmetry of order three. 


The upper symbol J means that the signs of the three outgoing 
branches coincide, [J and III means that the signs do not coincide, 
and in the case A of no rotation symmetry, the signs may alternate 
Cn) or not), 

The right or left arrow in the notations has the following mean- 
ing: consider the three outgoing directions as three points on the circle 
centered on the triple point. The cyclic order of the three visits of the 
corresponding branches defines an orientation of the circle. We use the 
right arrow if this orientation is positive and the left arrow if it is neg- 
ative. (A circle is positively oriented in the oriented plane if the frame 
(tangent vector, external normal vector) orients the plane positively). 

The three signs are the signs of the three segments, in the clockwise 
order of the outgoing branches starting at the left on the representatives 
of the triple points shown in table 2, the plane being oriented by the 
frame (to the right, up). 

The involution a, reversing the orientation of the plane, reverses 
the arrows, the signs of the segments and their order; the involution 
b, reversing the orientation of the front, reverses the arrows and the 
signs of the segments; the involution c reverses only the signs. The 
characters A and B, as well as the coincidence of the three signs and 
(in the case A) their alternative, are RO-independent. 


The stratum IJ of cusp crossings. The symbol C or D means that 
the orientations of the segment containing the cusp point and that of 
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metric 


Table 2. Local strata of the discriminant T’ 
RO-independent, RO-dependent and their symmetries 


the smooth one are concordant or discordant. 

The up or down arrow means that the cusp is of type up or down. 
(We orient the plane by the frame (to the right, up)). 

The first of the two signs is the sign of the segment going away from 
the cusp point, and the second one is the sign of the smooth segment 
crossing the cusp. 

The involution a, reversing the orientation of the plane, reverses 
the arrow and all the signs; the involution 6, reversing the orientation 
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metric 


Table 3. Local strata of the discriminant I 
RO-independent, RO-dependent and their symmetries 


of the front, reverses the arrow and the second sign; the involution c, 
reversing the coorientation of the front, reverses only the signs. Only 
the character C' and D are RO-independent. 


The stratum A of cusps births. The up or down arrow means that 
the two new-born cusps are both of type up or of type down. The sign 
is that of the smooth segment before the birth of the cusps. Involutions 
a and b reverse the arrow and the sign; the involution c reverses the 
sign only. 


Table 4. Local strata of the discriminant A 
RO-independent, RO-dependent and their symmetries 
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1.2. Coorientations of the local strata 


For every local stratum we define a quantity so that its variation (called 
elementary jump) at the generic crossing of the discriminant is always 
zero except at the crossing of that specific stratum. For the notations 
I follow [1] in the case of self-tangencies and triple points. The elemen- 
tary jumps not vanishing only at the crossings of the strata K and T 
are denoted by AJ and by ASt respectively. These jumps are locally 
constant functions defined along the smooth part of the discriminant. 
These functions are defined in table 5. For the elementary jumps at 
the strata II and A, I use the corresponding small greek letters Am and 
AX. All the indices (letters, signs and arrows) are the same as for the 
strata. 


The local coorientations of the strata are shown in table 5 (next 
page). They are independent of the orientations. The choice shown for 
a RO-independent stratum holds also for all the corresponding local 
strata dependent on the orientations. Moreover, for the strata K, II 
and A the corresponding elementary jumps are always chosen in terms 
of the variation An of the number of double points. For all the local 
strata of type T the coorientation is the same, and was defined in [2]: 
one has to look at the two triangles existing before and after the crossing 
of the stratum T, i.e., the triangles vanishing in the triple point. The 
cyclic order of their sides, given by the order of visit of a point running 
along the curve following its orientation, defines the orientations of the 
vanishing triangles. The sign of a triangle is s = (—1)? where q is 
the number of sides whose directions coincide with that given by the 
cyclic orientation. The signs of the two vanishing triangles are always 
opposite. The crossing of T is called positive if the sign of the triangle 
changes from — to +. The elementary jump in this case is defined 
by ASt = 2 (differently from the Arnold “strangeness” St of smooth 
curves, for which this jump is equal to 1). 

In the figures the local coorientation of a stratum is indicated by 
the normal vectors to the stratum pointing in the positive direction. 
The positive direction is defined by the corresponding elementary jump 
which is positive at the crossing of the stratum from the negative to 
the positive side. 


2. The 5 basic local RO-independent invariants 


2.1 Results 
Here we show how the basic local invariants independent of the orien- 


tations are defined. The proofs are in §§2.2, 2.3, 2.4. 
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STRATUM local coortentation elementary jump 


pa eee i = Ba 


At = 2 


(triangles rule) 


Table 5. The local coorientations of the strata of codimension one 


We impose the coorientability condition at the codimension 2 strata 
of the discriminant. 
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The set V of 12 numbers 


(evs, Via, Vip Vie Vip; 
ct Voto tl yotee ot VA) 


is the vector of the coefficients of the RO-independent jump 
6 VIF Ae VAN, 


This jump € defines a 1-cocycle if and only if the vector V verifies a 
system of linear equation 


MV =0 (I) 


called the coherence system. 


Lemma 2.1. The linear operator M is defined by the following matriz 


Th Se 0S 8) 0) 0) 0 0 0 
Qo0e i 1 Vl ala 0 O 0 0 0 0 
Cree 00 or i =i 0 0 
Cee 0. 0 Or 0. 1 0 0 -1 0 0O 
Oe Cm 0. Co 6 1 0 (a 
e020) Cet Cae 8 0 1 O -l O 
Cc OC 0 Ome 0 0) sl 
Oi O08. 0) OO G 0 Lil) Oi 0 
(eee) Ocal Oe il 0) 0 Oe 
6 0 0 0 1 Mel 10 0) 0 0 O 
o 0 0 0 elie GO: —1l DOE 0 0 
Lemma 2.2. The space of solutions of the coherence system has 


dimension 5. As the 5 R-basic solutions one may choose, for instance, 
the five jump vectors 


AJt =AJé + AJ§ 

AJ~ =AJg + AJp 

Sp = Anco + Amp + 2A(St, +Stis + Sta’ + st, + Sty) 
Ah=A(Jé + Jg +1) — Ap + Jp + 7D) 

Awe 
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(For example, the coefficients of & = AJ sare), — 1) Viel, all 
the other coefficients being zero). 


Definition 2.1. The following cooriented hypersurfaces, defined by 
the generators of the solutions of eq.(I) as linear combinations of the 
local RO-independent strata, are called basic RO-independent coori- 
ented strata: 


Tks IGS ae Te 

Ke kek, 

Tp Wert Wp 2 ea! ee eee 
H=Ki+Ket+llco-Kf-—Kp5-Wp 

A 


Lemma 2.3. The basic RO-independent 1-cocycles, whose jumps are 
defined in Lemma 2.2, are cohomologous to zero in the space Q4,,. 


Theorem 2.1. Every local invariant of fronts independent of the ori- 
entations and additive under the connected summation is a real linear 
combination of the following 5 basic invariants: 


Dies sl Si, 


whose jumps are defined by Lemma 2.2, and whose values on the basic 
curves are shown in table 6. 


Example. The number n of double points is a local invariant inde- 
pendent of the orientations. It is a linear combination of the basic 
invariauts 


1 
=h+—, 
n +5 


Corollary 2.1. Any generic path in Q;,9 connecting two L-immersions 
whose fronts belong to the front classes of wi, and Ki,9 (defined in fig. 8, 
see [1,2] ), contains at least i—1 events of type “self-tangency with 
parallel coorientations” and at least i—2 events of type “self-tangency 
with antiparallel coorientations.” If we want avoid the occurrence of 
triple points, then we will have at least 2i — 2 cusp crossings. If we 
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otf 
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Table 6. The value of the basic invariants on the basic fronts 


od 


k 
p= 2k WO, u 


want avoid the occurrence of cusp crossings, then we will have at least 
i —1 triple points. 


Example. In fig. 3 two paths connecting the front w3,0 with K3 0 are 
shown. In the path avoiding cusp crossings, there are two triple points 
events; in the path avoiding triple points there are four cusp crossings. 


PioomeOntthas iq) — JK eye" 2G), and] (@ao) — 
J ah) a e2) 4 Moreover, Sp(wi) — Sp(Kio) = 44-4. But 
Sp(wio) — Sp(Kio) = d2;(Agm + 2A, St) (summation on the events) 
by Lemma, 2.2. The triple points avoiding condition A;St = 0 implies 
my, Aj;m = 4i—4. Each cusp crossing contributes +2 to ae A,7. Hence 
the number of the cusp crossings is at least 27 — 2. Similarly, the cusp 
crossings avoiding condition implies that the triple points events are at 


least 2 — 1. 
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Figure 3. The fronts w;9 and K;,9 with their basic invariants 
and two paths from w3.9 to K3 0. 


2.2 The RO-independent 1-cocycles 
We give now the 


Proof of Lemma 2.1. Consider the strata of codimension 2 in the 
space of L-immersions . Some of these strata correspond to two non- 
generic events happening at different points of the front and others 
to the higher degeneration at one point. At a point of the first type, 
the germs of the discriminant hypersurface consist of two transversal 
hypersurfaces. Whenever the coorientations of both hypersurfaces are 
chosen, we obtain the coorientation of their union. Hence no coherence 
conditions arise at such strata. 

There are 8 types of codimension 2 strata corresponding to the 
higher degenerations at one point [1]. They contain the L-immersions 
whose fronts have one of the following singular points: 

KK: point of cubic self-tangency of the front; 
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TT: quadruple point; 

TIII: superposition of two cusp points (the tangents are supposed 
to be different); 

AA: singular point of order 5/4; 

KII: point of self-tangency coinciding with a cusp point; 

KT: point of self-tangency coinciding with a triple point; 

TI: superposition of a double point and a cusp point (the tangents 
are supposed to be all different); 

IIA: double point coinciding with a singular point of order 4/3. 

We require that the intersection index of a close path linked with 
every such strata having the codimension 1 discriminant, provided with 
the local coorientations defined above, vanishes. These conditions pro- 
vide the equations of the system (I). Every equation defines a row of 
the matrix M. To obtain all the rows we shall consider case by case all 
the eight types of strata. 

First we prove that the coherence conditions at the strata Ki 
TT, TM and AA are automatically satisfied and contribute nothing to 
the matrix M. 


Stratum KK. There are 4 types of strata KK because the two meet- 
ing branches K are of the same type (both of type Toe, aie or 
K>). In fact (see fig. 4) the situations at two generic points a and b are 
the same, looking at the parallelism or antiparallelism of the normal 
and the tangent vectors; moreover also the coorientations of the strata 
coincide in KK. The coherence condition at the strata K Ix is thus 
automatically satisfied. 


Stratum TT. It is not necessary to analyze all the possible cases: the 
following observation is sufficient to conclude that for this stratum as 
well the coorientability condition is automatically satisfied. Consider 
two generic points (a) and (b) lying on a stratum T symmetrically with 
respect to the stratum TT’ (see fig. 5). What happens at the triple point 
in the cases (a) and (b) is completely independent of the position of 
the fourth segment, so that the coorientations of Tin (a) and in (b) 
coincide, whatever the stratum T’ is. 


Stratum III]. To distinguish the strata of cusp crossing Hc and IIp 
one has to look only at the orientations (concordant or discordant). 
Then there are 2 distinguishable cases (see fig. 6) given by the two 
possibilities: (a) both cusps are of the same type (up or down) and 
(b) the cusps are of different types. Both cases give the same trivial 
equation 


Viee Vip vip — Vic = 0. 
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Figure 4. Local coherence at the strata KK 


Hence the coorientability condition is automatically satisfied. 


Stratum AA. Figure 7 shows that the two strata A meeting at the 
stratum AA are coherently cooriented. The two strata II are always of 
type IIc, and are also coherently cooriented. Again the coorientability 
equation is trivially fulfilled. 


Stratum KII. Here two local strata of type K and the two strata of 
type II meet. The K-strata are either both of type KT or both of type 
k~. However one of them is of type C' and the other of type D. In fact 
(see fig. 8) the segment crossing the cusp and successively tangent to 
one of its branches reverses its orientation moving from the left to the 
right in the figures (being vertical at the center, at the stratum KTI). 
Hence the sides of the narrower angles formed by the strata K and II 
at the point KII — the K-side and the I-side — are both of type C 
or both of type D. Since the number of double points is higher inside 
the two narrow angles, the coorienting vectors of the strata of type C 


Discriminants and local invariants of planar fronts 21 


Figure 5. Local coherence at the stratum TT’ 


point these angles inward while those of type D point outward. The 
two cases when the strata K are of type K+ (a) and of type K™~ (b) 
are responsible of the rows (1) and (2) of the matrix M. 


Stratum KT. The two local strata of type K are of the same type and 
their coorientations are of course coherent (the type of self-tangency 
does not depend on the position of the third segment). Hence the 
resulting equations involve only the two strata T. The coorientations of 
the two strata of type T are compatible [2], and they can be arbitrary 
since the cyclic order of visit of the triple point is mot fixed. The 
distinguishable cases are given only by the different types of strata 7’. 
They are shown in fig. 9. In (a) and (b) the types are the same, so the 
compatibility is automatically satisfied. In the other cases (ce), (de), 
and (f) we obtain the rows (3), (4), (5) and (6) of the matrix M. 
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(b) 
Figure 6. Local coherence at the strata Ill 


Figure 7. Local coherence at the stratum (ane 
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(b) 
Figure 8. Local coherence at the strata Kil 


Stratum TI. The two lines of type II in fig. 10 can be independent 
of any of the types IIc and Hp. Their types, as their coorientations, 
coincide in the two opposite branches. Indeed in two opposite points 
(for examples the points (a) and (b) in fig. 10) the types of the cusp 
crossing are the same. Hence this stratum gives again equations only 
for the jumps of the type ASt. 


The two branches of the strata T are coherently cooriented, as 
fig. 11 shows. In the four small triangles vanishing at the two triple 
points in (c) and (d) of fig. 10, the cyclic orders of the visits of the 
sides are the same. The orientations of two sides are the same. The 
orientation of the third side, being a cusp branch, is opposite at (c) 
and at (d) (fig. 11). Hence at the left of the strata T near (c) and near 
(d) the vanishing triangles have opposite signs. Similarly the vanishing 
triangles at the right of the strata T’ at (c) and at (d) have opposite 
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Figure 9. Local coherence at the strata KT 


sign. Therefore the coorientations in (c) and (d) are mirror-symmetric, 
i.e., they are compatible at the stratum TTI. 


The distinguishable cases for the strata TII are shown in fig. 12. 
In the cases (a) and (b) the RO-independent strata are the same. The 
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Figure 11. Coorientation of the two strata T near TI 


cases (c), (d), (e) and (f) give the rows (5), (7), (8) and (9) of M. 


Stratum JIA. The coorientation of the stratum A does not change at 
stratum ILA. The two strata of type IT are always one of type Hc and 
one of type IIp. They are cooriented in the opposite way (since the 
number of double points above the II strata in fig. 13 is higher than 
below it). 


The stratum of triple points cannot be of any of the types Tp and 
(be I The cyclic order of the visit of the triple point is fixed by the 
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Figure 12. Local coherence at the strata TIl 


orientation of the front. Hence the coorientation of the stratum T' is 
always as shown in fig. 13, i.e., compatible with the orientations of IIc 
and IIp. The four distinguishable cases are shown in fig. 13, where (a) 
and (b) are responsible for the rows (10) and (11) of M. 

Lemma 2.1 is proved. 


2.3 The cohomological triviality of the basic RO-independent 
1-cocycles 


Every solution of system (I) defines the jump of a 1-cocycle. This jump 
can be written in terms of the jumps given by the generators defined 
in Lemma 2.2. Every generator represents a 1-cocycle (basic cocycle), 
whose value at a generic closed curve is the sum of the corresponding 
jumps at the events along this curve. We now have to verify that these 
1-cocycles are trivial. 


Proof of Lemma 2.3. We have to prove that the value of any of the 
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Figure 13. Local coherence at the strata IIA 


5 basic cocycles on a generic closed path in the space of the marked 
L-iramersions vanishes. In fact every connected component a, 15 
(weakly) homotopy equivalent to a circle, so we have to do such a 
verification only for one noncontractile path. 

This is only a generalization to our 5 1-cocycles of the Arnold proof 
of the same lemma proved in [1] for the stratum K*. 

One has to distinguish two cases: 


Case 1. The Maslov index is different from zero. 

In this case, given a generic front in oF (4 # 0), consider the 
family of marked fronts obtained from it marking successively all its 
points in the order given by its orientation. This means that we have 
simply to move the front in the plane so that the marked point and the 
coorienting normal vector to it remain fixed (see fig. 14). 


When the turn of the front is completed, i.e., the front is again 
in the initial position, the tangent vector to the L-immersions at the 
marked point has done a number k = /2 of full rotations in the contact 


F. Aicardi 


SSOTYD 
CEYCRO 


Figure 14. A noncontractile path in 04,2 


plane, so that this family of marked L-immersion is a noncontractile 
path in OQ? ree This path does not intersect any stratum of the discrim- 
inant. Hence the values on it of basic RO-independent 1-cocyles are 
trivially vanishing. 


Case 2. The Maslov index is equal to zero. 

In this case generic noncontractile paths in 2° io are not so trivial. 
We verify that the sum of the jumps of the generators defined by Lemma 
2.2 in terms of the elementary jumps along a generic noncontractile 
path in the space (2; 9 for every value 7 of the index vanish. 

A generic noncontractile path in OQ? 6 is shown in fig. 15. 


All the generic crossings of the discriminant along this path are 
numbered (1-14), and the corresponding elementary jumps are indi- 
cated. The noncontractility is visible in the last part of the path, 
where there are no crossings of the discriminant but the marked points 
experience the same perestroikas as in the case ys # 0. Denote by (x) 
the value of the elementary jump at the crossing numbered by x. The 
increments 6 of the generators along the path are given by the following 
formulas: 


o( P= (6)- (10) — 0 

Eset 

5(Sp) = (3) + (5) + (9) + (11) =0 

oS) 6) O) SO Ce ae (a 
6(A) = (2) + (7) + (8) + (13) =0 


In the case of Q,~1,0, consider the path shown in fig. 16. 


From (a) to (b) the perestroikas are similar to perestroikas 1-14 
shown in fig. 15. The only difference consists of the n loops added 
near the marked point, which remain fixed and are not involved in 
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Se —2 Ax =-1 Anpr =? AJp =? J 5 
Dying = 
P-)-Q-O-S 
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10 
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a Se AM_=1 AJp_=2 ] 


Figure 15. A noncontractile path in Q1,0 


SOSm 


Figure 16. A noncontractile path in Qn-1,0 


these perestroikas. We go from (b) to (c), transporting the n loops 
beyond two cusps points. In fig. 17 it is shown that in the operation 
of the bypassing by a pair of cusps of the same sign of a loop, all the 
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Figure 17. The bypassing of a loop by a pair of cusps 


intersection indices with the discriminant vanish. 


The crossings of the discriminants are numbered and the values of 
the elementary jumps indicated. Using the same notation as before, we 
calculate the only possible non null increments at the crossing of the 
strata Tp and H respectively: 


O(8p) = (2) Pts) a (4) 0 
POS! 2 eee 


The other strata of the discriminant are not crossed in the path from 
(b) to (c). 

From (c) to (d) (see again fig. 16) the front is moved on the plane to 
the original position, so there are no other crossings of the discriminant 
in the noncontractile closed path. The lemma is proved. 


2.4 Definition of the basic RO-independent invariants 


The previous lemmas ensure that the functions Jt, J~, Sp, h and X 
define, up to additive constants depending only on the absolute values 
of the index and of the Maslov index, the basic local invariants of the 
fronts independent of orientations. This means that their variation 
along a generic path connecting two L-immersions does not depend on 
the path, but only on the initial and final L-immersions. 


Proof of Theorem 2.1 I) Existence of the additive invariants. 

We show that the choice of values of the basic invariants on the 
basic curves makes these invariants additive under the connected sum- 
mation of fronts. 
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A RO-independent local invariant X is additive under the con- 
nected sum of fronts if and only if it satisfies: 

i) X(0) = 0, @ being the front class of the circle with every orien- 
tation and coorientation. 

ii) X(a#G) = X(a) + X(G) for every choice of the generic points 
at the boundaries of the two fronts @ and @ where the gluing is made 
(when the orientations and coorientations allow this gluing). 

From (i) we obtain that w 19 has all the basic invariants equal to 
cro —— 2 and 2. 

From (ii) we obtain that wo has all the basic invariants equal to 


zero but J~ = —1 and h = 1 (see fig. 18). 
ee 
Cece coco YYY¥ 
6 — win = Woo F 400 => Wi 
J+ 0 0 0 0 
a 1 -2 =i -G@+1) 
Sp 0 0 0 0 
h 0 2 1 (i +1) 
NE 0 0 0 


Figure 18. The basic curves w;,0 


We obtain w;,o as the connected sum of (i+1) copies of wo,o- 

To define the values of the basic invariants on the curves w ,,, we 
proceed in this way. Start again from the curve ?, transform it into + 
and calculate its invariants (see fig. 19). Then write 7 as the connected 
sum of two fronts, both in the class of w1,2 up to the c involution. 


The invariants of the curve w),2 are thus defined, and also those 
of the curves w;,, obtained as the connected sum of k = 1/2 copies of 
wi. The values of the invariants for the basic curve w;,,, are finally 
obtained as the sums of the invariants of w;,9 and of w1,,, simce Wi, = 
Wio FF Wi1,p- 

To conclude the proof we have to verify that the condition (ii) 
holds for the basic curves. This implies that condition (ii) is verified 
for every sum of every fronts a and /. Indeed, suppose that p and 
gq are the points respectively at the boundary of a and § where the 
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Figure 19. The invariants of the curves w1,, 


gluing occurs. Choosing a point p at the boundary of a, it is always 
possible to transform the front a@ into another front having the same 
indices (in particular into the basic front) leaving the point p always 
generic and lying at the boundary and far from the points involved in 
the perestroikas (the boundary of a front is the boundary of the infinite 
domain of the plane belonging to the complement to the front into the 
plane). This is proved in [5]. Hence we can transform two fronts a 
and into the basic curves leaving p and q at their boundary, and glue 
the two basic curves at these points so obtaining a front which can 
be reached from a#( exactly by means of the same perestroikas (see 
fig. 20). 


Figure 20. The connected sum during the perestroikas 
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The validity of (2) for generic a and @ depends thus on the validity 
of (2) for the basic fronts. 

To transform this into a basic curve of type wi+j+1,,4v, a connected 
sum of two basic curves w;,,, and w;,, we have in general to do a set of 
perestroikas. We verify that the increments of all the basic invariants 
at these perestroikas agree with the rule of summation. The complete 
set of perestroikas is formed by the following elementary perestroikas 
and by the perestroikas obtained from these by the action of the RO- 
involutions (see fig.21): 

P,: bypassing of a loop by a cusp point (two cases fig.21-1a,b). 
In both cases one obtains the inverse perestroika by an RO-involution. 
Hence the values before and after the perestroika of an RO-independent 
invariant cannot be different. 

P2: bypassing of a double point by a pair of cusps of the same sign 
(fig.21-2). One has two opposite crossings of II¢ or of IIp, so all the 
values of the invariants remain unchanged. 

P3: bypassing of a double point by a loop (fig.21-3). In this pere- 
stroika, as one can verify from the figure, one has the following incre- 
ments of the invariants 


SPP) = =a yah aaa alah Oe 


P,: suppression of two cusp points of opposite signs by the creation 
of a loop (fig. 21-4). The nonzero increments are 


vs 6A = —2. 


Ps; annihilation of an internal-external pair of loops (fig.21-5). 
In this perestroika (see figure): 


6(J+) =2; 6&(J-)=4; &(Sp)=—-4; d(h)=-2; 6) =0. 


We are now ready to put together these perestroikas to transform 
every connected sum of basic curves into a basic curve. Consider all 
the 4 possibilities for the positions of the points p at the boundary of a 
basic curve: (1) between a loop and a cusp; (2) on a loop; (3) between 
two cusp points; (4) between two loops. In the cases (3) and (4) by 
means of perestroikas P; leaving p fixed, it is possible to transport all 
the cusps at one side and all the loops at the other side with respect to 
p, so going to the case (1). The invariants are unchanged, so we forget 
the third and the fourth possibilities for the gluing point. 
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Figure 21. Perestroikas occurring in the transformation 
into a basic front of a connected sum of basic fronts 


The choice of points p and q belonging to the basic curves uj, 
and w,;, can be of different types: type (1)-(1), type (2)-(2) and type 
(1)-(2). 

In the case (1)-(1), let the indices i and j of the two fronts be 
greater than zero (they must have the same sign to allow the gluing). 
After the gluing there are two possibilities (see fig. 22). If the Maslov 
indices of the two curves have also the same sign (a), we have obtained 
exactly the basic curve wj+;41,.4+, Whose invariants are exactly the 
sum of the invariants of the glued fronts. If not (b), let up = 2m > 0 
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and v = —2k < 0 be the values of the Maslov indices and moreover 
let k < m. Then near p and gq after the gluing there are two cusps of 
opposite signs. We do a perestroika P,, obtaining a loop, internal or 
external. By means of perestroikas of type P,, we transport this loop 
near the other loops (if any), i.e., far from the sets of cusps. We have 
again two cusps of opposite signs. With a perestroika Py we transform 
them into a loop, which result in their being of different types with 
respect to the previous one (i.e., external or internal). We do a number 
of these perestroikas equal to 2k obtaining k pairs of internal-external 
loops. To obtain the basic curve wi45+1,2m—2k we have to do k times 
the perestroika Ps. At the end of this process, the variations of the 
invariants which have occurred in 2k times P, and in k times Ps, are: 


6(J+) =2k; 6(J~)=4k —2(2k)=0, 6(Sp) = —4k;, 
6(h) =—2k; 6(A) = —4k. 


For every of such variations 6X (X = Jt, J~, Sp,h, ) we verify easily: 


X(wiam) + X(wyon) + OK) =  X(witj41,2m-2k) 
a —m =) 2k —(m—k) 
J~ —-(i4+1) —(j +1) 0 —(i+j+2) 
aye 2m Ds —4k 2(m — k) 
h i+t1l—m jg+1—-k 2k; i+j+2—-—(m—k) 
r 2 Die —4k 2m — 2k 


In the case (2)-(2), the signs of the indices still have to be equal 
to allow the connected sum (see fig. 23). This case is reduced to the 
previous one by means of type Py perestroikas and of a perestroika 
where one has 


The basic front so obtained has index equal to i+j—1. This agrees with 
the differences 6J~ and 6h with respect to the sum of the invariants: 


J (wiam) + J7 (wj,2k) — J” (witj—-1,2m42k) = —2 
h(wi,am) + A(w;,2k) — R(wi+j—1,2m+2k) = 2 


In the case (1)-(2) the signs of the indices are opposite. Let their 
values be i and —j and let —j be the index of the curve where the 
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(a) 


= Pes 


(b) 


Figure 22. Connected sums of type (1)-(1) of two basic fronts 


gluing point is of type (2). Moreover let i > j. Suppose that the 
Maslov indices are both positive (see fig. 24). 


We transport the 7 loops and the 2k cusps as shown in fig. 24 
making exactly 7 times the perestroika P3; and k times the perestroika 
P,. We obtain a curve with w+ v cusps of the same sign, 1+ 1 external 
loops and j internal loops. By means of perestroikas of type P, we 
assemble together the cusps and the loops. Then we eliminate the 7 
internal loops with j external loops by means of 7 perestroikas of type 
Ps. Since the Maslov indices have the same sign, we have obtained a 
basic curve with index 71 — j7 + 1, which has indeed i — 7 + 2 double 
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Figure 24. Connected sums of type (1)-(2) of two basic fronts 


points. The total variations of the values of the invariants are 


Cape) = 0; 85) = 27 + 49 = 23; 
6(Sp) =4j7 43 =0; 6(h) = 27; 4(A) =0. 
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These increments coincide with the differences between the values of 
the sum of the invariants of the added basic curves and the values of 
the invariants of the basic curve w;_j41,.4v, aS Shown: 


X (Wi,2m) ar X (w_j,2%) + 6(X) = (i aero) 
IB —m —k 0 —(m +k) 
J~  —(t+1) —(j +1) 27 —(i-—j +2) 
Sp 2m 2k 0 2(m +k) 
h t+1l—m j+1-k —2) t1—gt+2-—(m+k) 
aN 2m 2k 0 2m + 2k 


The case 7 < 7 is similar. If the Maslov indices do not have the same 
sign, then we come back to the situation of the case (1)-(1). The 
existence of the additive invariants is proved. 


II) Uniqueness of the additive local invariants. 

An additive invariant is unambiguously defined by its jumps (see 
Lemma, 3.6 below). Hence we have listed all the additive local RO- 
independent invariants. 


2.5. Some remarks on the RO-independent invariants 


Remark 2.1. The nonexistence of the invariant St in the case of the 
fronts is due to codimension 2 stratum ITA, showing that the stratum 
of triple points is not coorientable. 


If we start from a smooth front in (;,9 and we forbid the crossing 
of the stratum A, we are moving just in the connected space of the 
smooth immersions of a circle in the plane: the invariant \ is zero, the 
invariant Sp becomes the invariant St (up a constant factor) and the 
invariant h becomes n. 

If now we forbid the crossing of the stratum A starting from-a 
generic front with a given number of cusps (and Maslov index equal or 
not to zero), we obtain for the elementary jumps of the local invariants 
a new coherence system M’V = 0, where M’ is the linear operator 
obtained from M erasing the last two rows, given by the coorientability 
conditions at the codimension 2 stratum IIA. The new system has the 
set of solutions of dimension 7, i.e., the two basic cooriented strata ID 
and H split in two different cooriented strata, respectively: 


P27, oy i a 


T 
Py Te + Hp 
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and 


nfo =Ki+kKet+llc 
Hp =Kfi+Kp5+Ub 

The cohomological triviality of the corresponding 1-cocycles has 
however to be proved, because the homotopy type of the space of L- 
immersions whose fronts have a fixed number of cusp points is not 
investigated yet. 

It is interesting to note that, in the hypothesis that the 1-cocycles 
defined by the jumps at H¢ and Hp are trivial, the corresponding 
invariants hc and Ap should be another generalization for the fronts 
of the invariants J* and J~ defined for the immersions of a circle in 
the plane. 

Their definition in fact is totally independent of the coorientation 
of the front. Moreover the local coorientations of the strata Hc and 
IIp coincide with the coorientations of the corresponding strata Ke 
and K e so that the invariants hc and hp, forgetting the coorienta- 
tion of the front, should operate as a sort of “smoothening,” i.e., their 
values should coincide with the values J* and J~ of a corresponding 
immersion where all cusp points are smoothened. 

This is the meaning of the identity Ah = An when 4 is fixed, where 
h = hc —hp (corresponding to J+ —J~ for the immersions). Moreover 
these conclusions should hold also for the noncooriented fronts with any 
number (even or odd) of cusps. 


Remark 2.2. The strata K+ and K~ are not irreducible [1]. Indeed 
one can associate to every self-tangency event a nonordered pair of 
numbers (called the partial indices [1][5]) which is invariant along the 
components of the strata. The local invariants J + and J~ are the sums 
of an infinity of invariants, defined in the same way (thus independent 
of the orientations), but indexed by the partial indices. This fact does 
not contradict Theorem 2.1 since these invariants are not additive under 
the connected summation of two fronts. 


3. The basic RO-dependent local invariants 


3.1. Results 
The proofs of the following facts are in §§3.2, 3.3, 3.4 and 3.5. 

In this part we repeat the procedure followed in the search for the 
RO-independent basic invariants starting from the elementary jumps 
dependent on the orientations, defined in Lemma 1. Their number 
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is 54, so it is not convenient to write the entire system of the local 
coherence equations at the codimension 2 strata. 


Definition 3.1. A partial coherence system is a set of equations 
belonging to the entire coherence system involving only the jumps at 
the strata of only one type (K, T, II or A). Its solutions are called 
reduced elementary jumps. 


We find first a set of reduced elementary jumps, then for these new 
variables we write the other equations of local coherence at the strata 
of codimension 2 involving different types of jumps. 


Lemma 3.1. The following reduced elementary jumps form the fun- 
damental system of the solutions of the union of the partial coherence 
systems for the strata K, T and II. 


= 


) AJ, 
) AIg_ 
i Ae OIE te AJ5 
4) AJg =AJg,+AJo. 
) 
) 
) 


bo 


AJp. ae DOG. ay ASD 4 i 
— MIG. + Adp_ 
ae aZ aps 
Bova yi Asc, Mee SAGE 
a,8,7.7 a,8,n 
Moo = Ang; + Atpy 


yt 


) 
) Any” =Angy + Anpt 
) ra = Angy F Anh, 
) An; — Ang, + ee 
12) Ao = Angi + Arpy 
) Amy” =Angy + Anpr 
) Ne = Any oo Andi 
) Gay = Anéy =e Anh 
) 


Are = SS NE, 
a8, 


where the indices a, 8, y run on the set of signs (+,—), and n runs 
on the set of arrows. 
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Every reduced elementary jump defines a 1-cochain whose value at 
the 1-chain positively crossing transversally the corresponding stratum 
of the discriminant is equal to 2, being zero if the crossing takes place 
at any other stratum. 


Definition 3.2. We define some linear combinations of reduced jumps 
of the above list, which will be useful in the sequel: 


Min = 3 Ane, = SS (Sage —Ato 
a,8,n a8. 


1+ _ +2 
Ar, = Ne 
B 
= —p 
(a = Ave 
B 
rn +6 
Ni a, 
B 
aay —B 
Ar, = ea 
B 


Art = Arg — ye Air, © 
ot) 
An?- =Anco— > Anzt 


a,7) 


where a and G run on the set of signs (+,—), 7 on the set of arrows 


(T, 1). 


Remark 3.1. Amc and Arp are the elementary RO-independent 
jumps defined at the strata of the cusp crossings. 


Ane is the jump at the cusp crossing strata which is nonzero only 
if the cusp is of type up and the branch going away from the cusp point 
is positive. Analogously for An; (Ane Ar\”) the jump is non zero 
only in the cases (negative, up), (positive, down) and (negative, down) 
respectively. 

Ar2+ (An?~) is the jump at the cusp crossings strata nonzero only 
if the smooth segment crossing the cusp is positive (negative). The 
jump An?* (A7r?—) has the opposite value on the positive crossings of 


the strata Ig and Ip. 
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Remark 3.2. There are no partial coherence systems for jumps of type 
AX, so we have to add to the set of 16 independent reduced elementary 
jumps all the 4 elementary jumps of type AX. The 54 unknowns in 
the system of the coherence equations are thus reduced to the 20 new 
unknowns. 


Definition 3.3. For every reduced jump (and for every linear combi- 
nation described in definition 3.2) we define the reduced local stratum 
to be the linear combination of the local strata corresponding to the 
elementary jumps which compose this reduced jump or combination. 


To write the matrix of the coherence system we denote the unknown 
jump € (defining the 1-cocycle) by 


C= VIN hee ye ANE 
and denote by V‘") the vector of coefficients: 


WAT aE Se elon Wi Vole, Wola Voli, Wet 
Vos, Vay, Vis Ng Vin VT, Va Vay, Vic. 
Vala WAL, VAL; vom 


Lemma 3.2. The vector A‘) defines a 1-cocycle if and only if it 
verifies the coherence system 


Mery =0 (IT) 


the linear operator M°" being defined by the following matrix 
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maooo°o 


= 
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i=) 


= 


=I 


0 0 


The fundamental system of 1-cocycles derived from the solutions of 


eq.(II) is formed by the following baste jumps: 


+AJf 


+ 
C= 
+ AJ pePA TS _ 


+ AJ, 


SP PS 


Cc 


Ne 
NSp = NSE Panga Ap 


Ne 


L 
aie 


_tAnr++ Ar +d 


D 


py = 2h i emmee 24, 


A 


ae re 


De 
bgp = ise — ie 


ie 


IS Are — Nae 


1 


AM =AAl + Ad 
Ad = AM. + Ad 


{ 
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Remark 3.3. AJ*t, AJ~ and ASp are the jumps of the RO- 
independent basic invariants (Theorem 2.1), so in the following we 
analyze only the other six new possible invariants. 


Lemma 3.3. The RO-dependent jumps defined by Lemma 3.2 satisfy 
the following RO symmetries: 

Aft and Af~ are Rab) (bc) invariant. Indeed the involutions a, b 
and c interchange the values of these jumps. 

Ap! and Ap! are invariant with respect to Rate). Indeed the tn- 
volution 6 interchange the values of these jumps, while the involution c 
reverses the signs of the values of these jumps. The involution a acts 
as the involution bc. 

AX and Ad" are Rad)(c)-tnvariant: the involutions a and b inter- 
change the values of these jumps. 


Definition 3.4. The following cooriented hypersurfaces, defined by 
the solutions of (II) as linear combinations of reduced local strata, are 
called basic RO-dependent cooriented strata: 


= = 2 it J 
FY —2KG, —2K5, +07 +A, +A4 
2 ee ey er 


i Ue 
[Xe NF ee 
AT=Al + at 
Boe Ne 


Lemma 3.4. The basic 1-cocycles defined by the jumps Aft, Af-, 
Ap!, Ap', AX! and AX are cohomologous to zero in the space of L- 
immersions . 


Theorem 3.1. Every local invariant of the fronts additive under the 
connected summation is a linear combination of the RO-independent 
invariants, of the Maslov index and of the following 6 basic invariants: 


(oe meme mr LAS 


whose jumps are defined in the lemma 3.2, and whose values on the 
basic fronts are shown in table 7. 
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Table 7. The basic fronts with the values of their 
RO-dependent basic invariants 


basic curves 


Com 
Com 


Remark 3.4. The basic fronts with k # 0 are asymmetric. Hence 
the action of RO transforms each of them into 8 fronts of 8 different 
classes. In table 7 only 4 classes are shown, distinct by the values of the 
indices. In fact to every of such classes w;,, there correspond another 
one with the same values as the indices, namely the class abcw;,,, (for 
instance, the class w;,,, shown in table 6). By Lemma, 3.3, the class 
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abew;,, has the same values as the invariants of the class w;,,, but f os 
and f~, which are interchanged. 


On the other hand, the class acw;,,,, having the opposite signs of 
the indices, has the same values as all the invariants of the class Wj mu. 
In fact the involution ac preserves the values of the invariants f* and 
f7~ and interchanges the values of p! and p!, and the values of \! and 
of \+. But these interchanged values on all the basic curves coincide. 


Corollary 3.1. There are exactly 10 (including the Maslov index) basic 
local additive invariants of the fronts. 


Proof. The 10 invariants are the Maslov index, the 6 RO-dependent 
invariants of Theorem 3.1 and the invariants J+, J~ and Sp, by Lemma 
3.2. The other RO-independent basic invariants of Theorem 2.1 can be 
written as: 


h=ft+fro-J?+J°- =x 
A=ATH AL. 


Corollary 3.2. To reverse the coorientation of a basic front having 
Maslov index equal to , one has to make at least |u| cusp crossings. 


Proof. The difference of the values of the invariant p = p' + p! on two 
fronts with opposite coorientations is 24. At every cusp, crossing the 
invariant p changes by +2 or —2. 


Remark 3.5. Unfortunately Theorem 3.1 does not allow us to prove 
the Arnold conjecture that one of the two curves of fig. la cannot 
be transformed into the other without crossing of the discriminant of 
the parallel coorientations self-tangencies K+ if the cusps births are 
forbidden. They are indeed distinguished only by the values of the 
invariants f* and f~. Different paths connecting the two curves in 
Q1,9 are shown in fig. 25. 
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Figure 25. Examples of the F* and F~ discriminants. The values of f* 
and f~ are shown in the connected components of the complement 


In table 8 a list of oriented and cooriented fronts in the oriented 
plane with the values of their 10 local additive invariants is presented. 
The action of RO on these fronts gives the representatives of all the 
front classes with at most two double points and at most two cusp 
points. Their invariants are obtained applying Lemma 3.3. 
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Table 8a,b. The additive local invariants of fronts 
with at most two double points and at most two cusp points 
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3.2 The RO-dependent 1-cocycles 


We solve the coherence system at the strata of codimension 2 for the 
RO-dependent local strata in two steps. In the first one we give the 


Proof of Lemma 3.1. We write the partial coherence systems for the 
reduced jumps of type AJ, ASt, Ar. 


Stratum KK. For the reduced jumps at the strata of self-tangencies, 
the partial coherence system is 


MucxyVJ =0 (IID) 


where VJ is the vector of the coefficients of the RO-dependent basic 
jumps defining the jump 


C= Vig, Ade, +a Vigo, ape 
namely 


VI =(Vioou, Viegas V Ip es Vibe Vian Vee 


VJ54.e: VI b+ VI D_,e: VI D- 4) 
and 
yo =a. oO 8 @ 0 6 Oo 
a 00 00 ee oon to 
Co) SIT ct ow tT al Oo 
Yue o COO OO 1 =f 


There are 6 different cases involving the 10 local RO-dependent 
strata giving the coorientability conditions at the strata KK of codi- 
mension 2. They are shown in fig. 26. In (a) and (b) the coorientability 
condition is automatically satisfied, (c), (d), (e) and (f) give respec- 
tively the rows 1-4 of Mix). The generators of the solution of the 
system (III) consist of the reduced jumps denoted 1)-6) in the state- 
ment of Lemma 3.1. 


Remark. We remark that the reduced jumps AJg and AJ}, are RO- 
independent, while the other four jumps are Rp) (p-)-invariants. 


For the jumps at the strata of triple points, we have to solve the 
partial coherence system for the vector V(St) of the coefficients ot the 
24 elementary RO-dependent jumps, 


McxrrmV St 1 (IV) 
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Figure 26. The RO-dependent strata KK 


Indeed the coorientability conditions at the strata KT and TTI involve 
nontrivially only the strata of type T. The coorientations of the two 
branches of these strata meeting at the strata of codimension 2 are com- 
patible (see Lemma 2.1). Hence all equations of the system (IV) come 
from equations of the type VSt(1) — VSt(2) = 0, ASt(1) and ASt(2) 
being the elementary jumps of type ASt at the particular local strata 
of triple points consisting of the branches T(1) and T (2), respectively). 


Stratum KT. We first solve the system given by the conditions at 
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strata KT 


Mi ME VSta 
Mr) VSt = =i 
Me Mi ViSts 


where 


WGA =(V Si, Voli V oiy og, VC 


A—? as 


WiSae VGii, Vota , Vogal 


A=) 


Vista VS a oa amy cae 


Ac—? A—? A? 


ViSt 22 VSE) ay ciel, Sees | 


A— >! As 


Wiss ST VS Gis es 


B—) Be—? Be—? 


VStg1., VStRL_, VStg_, VStg) 
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(rV") 


are the coefficient of the jumps (ASt,**,..., ASt3_*) respectively, 


Alo 9 


MB is the zero matrix, M 4! is the identity matrix, while 


Moe (0 a) eee 0s estes Ce) es) eon) 0 
iO sae Oe Oe a0 OO) 0 
OO Otte Ose OO Ao 0 6 0 
OL GC) OOS UO Gl 
Ce eC ee OO) Oe Oe Oat) 0 
UP OOO SO 0.0: OF 10 0 se 0 
CeO OS OOS 0: 07 Oe 0 et at 0 
C0 50050 0 Oe arr ew 
US OPO SO OOO Sao 0 
a OO a Te a eel ee ee) 0 
OO 0. 0 Ora > 0 Cea ea 0 
CTO 0 OC Oia Oo OO 
Oy 0 0. 0) 20 Us Os OCS Oe eel) 0 
O° Oo 0 0 0 CORO te 0 SU 0 
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Oi 0 0 Oe CO 

TE 1 i AO 
ye = C00 SOS Oe tO 
CeO) 0 FO 0a 0 1 

OO 0) Ot. 0 

CO 0 0 

Cc CSU i 

0 On OO 

0 oo 0 0 0 Se! 20 
(CO) a Uae 


The equations (IV’) come from different cases shown in fig. 27. 


In cases (a) and (b) T(1) and T(2) are both of type T'4: the cyclic 
order is reversed because the second and the third segments are in- 
terchanged (a), or the first and the second ones (b). We have from 


(a) 

Toe = 1 
and from (b) 

wus — GE = 0 


where if 7 is the symbol “—” then —7 is “«—” and vice-versa. These 
equations correspond to the first 14 rows of M xt), i-e., to the rows of 
Mi. 

In the case (c) T(1) and T(2) are of type T, and Tg. The cyclic 
orders on both strata are the same. Hence we obtain the successive 16 
rows of Mx) (i-e. the rows of Mi.) in the form 


= 
V ste” — VStg] = 0 


(where in the case Tp the three signs a, 3, have to be considered up 
to cyclic permutations). 
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The space of the solutions of the system ({V’) has dimension 4: we 
choose the first reduced jumps: 


Om +++ 
Noi) ces 
Zin 
AG = ING 3 Ss Nie 


oe 


ASt? = Ea 94 Dasias 


ASB = Sasi 


where Z runs on the set (A, B) and o runs through all the permutations 
of the three symbols. 


Stratum TII. For the jump ASt = ee VSt' - ASt; we write now the 
partial coherence system 


MrmVSt = 0 Civ) 


where V St is the vector of the coeficients VSt;, and 
1 
Mam={(0 1 -1 0 
0 


Indeed it is sufficient to note that to the strata T'(1) and T'(2) meeting 
at the stratum TTI there always correspond different types of triple 
points. The types of jumps ASt? are distinguished only by the number 
2 of segments of sign + which cross at the triple point. Two of these 
signs are fixed; the third, being the sign of one of two branches of a cusp, 
changes from T(1) to T'(2) (see fig. 28). So there are 3 possibilities for 
the pair (474-1) being 7 = 0) 12,3, 


The matrix Mipny has rank equal to 3, so the solution we obtain 
for the reduced jumps at the strata of triple points has dimension 1. 
Therefore the reduced jump ASt is the sum of all the 24 local elemen- 
tary RO-dependent jumps. 


Stratum III. For the reduced jumps at the strata of the cusp cross- 
ings, the partial coherence system is 


Mam VI =0 (V) 
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Figure 27. Local coherence at the strata KT 


where 


Vi (Vag; Vitor Viney tone Ve VT» Vag Vaéy 


Vapr> Vit» Vi5w Vabr Vane. VT)» Vabl, Vrb)) 


is the vector of the coefficients of the corresponding basic jumps 
(Aree in tans, ) and the linear operator My) is defined by the 
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Figure 28. Local coherence at the stratum T'II 


following matrix 


0 0 tl =i @ 0 0 0 0 0 i =-l @ 0 0 0 
iL Oo =! @ 0 0 0 0 oO =i @ i 0 0 0 0 
oO =I @ 1 0 0 0 0 1 Oo =i @ 0 0 0 0 
=i il 0 0 0 0 0 GO el 1 0 0 0 0 0 0 
0 0 0 0 0 0 = 1 0 0 0 0 0 =i i 
0 0 0 0 0 1 O = 0 0 ) =l @ 1 0 
0 0 0 O@ =1 @ IL 0 0 0 0 0 0 1 OO ll 
0 0 0 0 to =i @ 0 0 0 0 0 l =I 0 
0 0 1 0 0 Oo =1 @ 0 0 ee) 0 0 iL 
1 0 0 0 0 0 0 -l1 0 -1 QO 0 0 0 1 0 
0 0 0 tL =i © 0 0 0 0 =I w 0 1 0 0 
0 1 0 0 Oo =-l @ 0 =l @ 0 0 1 0 0 0 
0 0 Oo ol @ 0 0 1 0 0 1 0 0 0 = 
0 0 =l @ 0 1 0 0 0 0 0 io =i @ 0 0 
Oo = wo 0 0 0 1 0 1 0 0 0 0 0 O = 
a) 0 0 1 0 0 0 0 it 0 0 0 =l @ 0 


The equation (V) is obtained at the strata III]. Consider first the case 
of two cusp points both of type up. There are 4 cases depending on the 
coorientations of the cusp branches. The two cases (a) and (b) of fig. 29 
give respectively the rows 1 and 2 of the matrix Minn). Reversing the 
coorientation of the front, the two signs of every elementatry jump are 
reversed while the other symbols remain unchanged. Then one obtains 
from the previous ones the cases (d) and (c), giving the rows (4) and 
(3) respectively. 

Consider now the case when the two cusps are both of type down. It 
is obtained applying the involution b to the previous situations. Every 
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Figure 29. RO-dependent strata III 


elementary jump changes in this way: 


so that we obtain rows 5-8 of Mian) from rows 1-4 reversing the signs 
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of @ and the direction of the arrows. 

Two of the four cases when the cusp points are one of type up and 
the other of type down are shown in fig. 29: (e) and (f) give respectively 
the rows 9 and 10 of Mim). Applying the involution c to these two 
cases one obtains the cases (h) and (g) and the rows 12 and 11. The 
other 4 cases obtained applying the involution b to the four preceding 
cases (the stratum III is not invariant with respect to this involution), 
give the rows 13-16. 

The action of the involution a on this stratum is equivalent to the 
action of the involution bc, so there are no other cases. 

The rank of the matrix M qm) is equal to 7, so that we choose the 
9 reduced jumps (8)—(16) of type Az as defined in the statement of 
Lemma, 3.1 in terms of the elementary RO-dependent jumps. 


We are now ready to give the 


Proof of Lemma 3.2. We have still to consider the strata AII, AA 
and ITA of codimension 2. 


Stratum KII. The coorientations of the local strata of self-tangencies 
and of cusp crossings meeting at this stratum and their characters C 
and D have been discussed in the RO-independent case. In the RO- 
dependent case there are 16 distinguishable situations. Indeed, consider 
the cases (a) and (b) of fig. 30. They are distinguishable also in the 
RO-independent case. We note that in the two occurrences of the cusp 
crossing along the closed path linked with the stratum KI], the types 
up or down of the cusps are the same as the signs of all segments. 
Therefore the two elementary jumps of type Az have coincident upper 
signs and arrows. In terms of the coefficients of the elementary jumps 
of type Az the cases shown in (a) and (b) give: 


Vie, = We, = Yang, eV = 0 
Yuet — Use — Tha Vl. =O 


So we obtain rows 1 and 2 of the matrix M°", since in terms of the 
linearly independent reduced jumps 


Wie: = Wi = Wie — Vite +Vr1eo. 


We find the other equations applying the RO-involutions to these 
two cases. We notice that such involutions do not change the upper 
signs of the jumps of type AJ, and do not change the characters C and 
D of the strata. 
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Figure 30. Four examples of RO-dependent strata KII 


For example the cases (c) and (d) in the figure, which give the rows 
2 and 4 of the matrix M°’, are obtained applying the involution c to 
the two preceding cases. The result of this involution reversing all the 
signs of the segments is the following: for the jumps of type AJ the 
lower signs are reversed, while for the jumps of type Az the upper signs 
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are reversed. 

If one applies the involution 6, again the lower signs of the jumps 
of type AJ are reversed, while only the second upper sign (the sign of 
the smooth segment crossing the cusp) is reversed in Ane? . Moreover 
the character up becomes down: 


Are — Ain (26.8) 


The involution a reverses the lower signs of the jumps of type AJ 
and all the upper signs and the arrow of Az. Therefore the involution 
ab leaves unchanged the jumps of type AJ and the arrows of the jumps 
Az, and reverses only the first of the upper signs of Az: 


Mer — Mg = 1D) 
i.e., for the reduced jumps 


Ar: me Ar, oe 
In this way one obtains the rows 5-8 of M°" from the first 4 rows. 

Moreover it follows that the involution ac only reverses the arrows 
of the reduced jumps at the cusp crossings. One obtains the rows 9-16 
of the matrix M°" applying this involution to the rows 1-8. 


Remark. We have chosen the closed path linked with the stratum 
KII such that the crossing of the local stratum of type Tg is always 
positive (see fig. 30). Hence the value of the coefficient of Vic is equal 
to 1 in every equation. 


Stratum AA. The two strata of type A and those of type II are all of 
type up or all of type down; moreover, they are cooriented always as 
shown in fig. 31. 


The two strata II can be only both of type C, as we remarked in the 
case of RO-independence (§2.3), where there is only one situation. In 
the RO-dependent case, we note that the situation is always invariant 
with repect to Ray). Hence there are four distinguishable cases, shown 
in fig. 31. In the case (a) one has 


VAL + Vande — Vagt — val =0 


We can reverse the orientation and the coorientation of the front, so 
obtaining the other 3 cases. Reversing the orientation, the character 
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(d) 
Figure 31. The RO-dependent strata AA 


up becomes down and all the signs are reversed. Reversing the coorien- 
tation, only the signs are reversed while the arrows remain unchanged. 
One has always: 


VA + ve? Vane PVA, = 0 


Remark. The coefficient of Vic is always trivially equal to zero. 


The cases (a) and (b) in the figure give the rows 17 and 18 of M°", 
while (c) and (d) give the rows 19 and 20. 


Stratum IIA. The two local strata A separated by the stratum IIA 
are of the same type and coherently cooriented. The two strata II non 
coherently cooriented are always both of type up or both of type down, 
and one of type C and the other of type D. The coorientation of the 
stratum T is consistent with the coorientations of both strata I] (see 
§2.3). Consider the example of fig. 32. 


The two strata of cusp crossings at the points (a) and (b) are es 
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Figure 32. A RO-dependent stratum IIA 


and Ip]. Hence the corresponding equation is: 
++ —+ _ 


Changing the orientations and the coorientations of the segments one 
can obtain every local stratum in (a), say Me At the same time the 


stratum in (b) becomes n°" , (where if Z = C then Z = D and vice 


versa). Therefore for every pair of signs (a, 3) and for every arrow (7) 
there is a situation where the nonzero coefficients of the reduced jumps 
on a closed path linked with the stratum IIA satisfy 


Vane? —VSt+Vn,* 98 + Vac =0. 


Taking into account the symmetry of the equation (a, 8) — (—a, —f) 
we find 4 distinguishable cases. These four cases give the rows 21-24 
of the matrix M°". 

To complete the proof of the lemma we have only to solve the 
linear sysem (II). The rank of the matrix M°" is equal to 11, so the 
dimension of the set of solutions is 9. The generators we choose satisfy 
(II) and are linearly independent. 


3.3 The symmetries of the RO-dependent basic jumps 


Proof of Lemma 3.3. To prove this lemma we have to study the 
action of the group RO on the reduced jumps (of which the basic jumps 
are linear combinations). 

ihe reduced jumps Ave. Ads , and Ar?* are transformed into 
TING ae AJp_ and An? by the involutions a, b and c. Indeed these 
involutions reverse the signs of all segments of the front. 
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The reduced jump ie becomes Am!* under the action of the 
involution b. Indeed the character up is changed into down and vicev- 
ersa while the sign of the cusp branch going away from the cusp point 
does not change under this involution. The involution c, reversing this 
sign, transforms Az!* into Am!—. The involution a acts on Am as be. 

The reduced jump Ao is transformed into AA by the involutions 
a and b. Indeed each of these involutions reverses the local orientation. 
The involution c, revrsing only the coorientation, transforms AML into 
AK. 

The symmetries of the basic jumps follow from the symmetries of 
the reduced jumps which constitute them. 


3.4 The cohomological triviality of the RO-dependent 1- 
cocycles 


Proof of Lemma 3.4. To prove this lemma, we have to verify that 
the value of every basic 1-cocycle on every closed path in ();,,, vanishes. 
We proceed exactly as in the proof of the corresponding lemma in the 
case of RO-independence, doing this verification for a representative of 
the unique homotopy class of noncontractile paths in the space of the 
marked L-immersions for every values of the indices 7 and p. 

In the case pp # 0, there are noncontractile closed paths in 2? , such 
that there are no crossing of the discriminant at all (see for example 
fig. 14). 

In the case (44 = 0, i = 1) consider the path shown in fig. 15, 
where all the 14 crossing of the discriminant are numbered and the 
corresponding RO-dependent elementary jumps are presented. The 
sum 6 of the values of the basic RO-dependent jumps along this path 
can be easily calculated (the value of the jump at the event x being 
denoted by (x)): 


( —(5) + (7) + (8) + (9) + 2020) = 6; 

6(f-) = —2(1) + (2) — (3) — 2(4) + (11) + (13) — 2(14) = 0; 
( ) + (5) 
( 


s 
+ 
ll 


6(p') = —(3) + (5) =0; 
6(p') = (9) — (11) = 9; 
6(A') = (7) + (13) = 0; 
6(A+) = (2) + (8) =0 


In the case of 2; with i # 1, consider again the paths shown in 
fig. 16. The values of the basic 1-cochains on these closed paths are still 
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vanishing if they are vanishing on the part of the path consisting of the 
bypassing of a loop by a pair of cusps of the same sign. The crossings 
of the discriminant and the corresponding elementary RO-dependent 
jumps are shown and numbered in fig. 17. We can thus calculate (with 
the same notation as before) the values of the 1-cocycles on the closed 
paths 


5(f*) = —(1) — (2) + (3) + (4) =0; 
é(f~) =0; 

fei GB) =0, 

SP = 2) = (2) = 0 

[CE 10, 

HO = (0, 


For the paths in 2+; +,, one obtain the same results acting on the paths 
shown in fig. 17 by the involutions b and c and applying Lemma 3.3. 


3.5. Definitions of the basic RO-dependent invariants 


Proof of Theorem 3.1 I) Existence of the additive invariants. 

Lemma 3.4 ensures that the basic RO-dependent jumps given by 
the solutions of the eq. (II), define RO-dependent local invariants in 
the connected components of the space of L-immersions up to additive 
constants which are still arbitrary. We shall now choose the values of 
these invariantss on the basic curves, making these invariants additive 
under the connected summation of fronts. 

An additive RO-dependent invariant X has to verify: 

i) X(@) =0 on the four front classes (6, a0, cO, acO) of the circle 
(which are R(,,)-invariant); 

di) X(a#6) = X(a) + X(G) for every generic points of a and 8 
where the gluing takes place (when the orientations and the coorienta- 
tions allow it); 

Uji Myf f= ple’, A! A!) are the valiica of the 6 
invariants on a generic front y, then the following symmetry conditions 
have to hold (by Lemma 3.3): 


SG pl et 
X(ey) sf fF ap pl 
A (Oy eee ef ea, ee ee ee 
Key Fpl Sa 


In the case 4 = 0 we obtain the values of the invariants of Wo 
(i > 0) asi+1 times the values of the invariants of wo,o- Indeed w;, is 
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the connected sum of 7 + 1 identical copies of woo. The values of the 
invariants on the curve woo are found by (i) (see fig.33). 


eal 
6 — win = 0 # 0 > Yio 

f+ ‘0 4 2 2(i +1) 
f-~ 0 0 0 0 
p' 0 0 0 0 
p- 0 0 0 0 
ae) 0 0 0 
A 0 0 0 0 


Figure 33. The RO-dependent basic invariants of w;,o 


Remark 3.6. Every curve w;,9 has an R(qj)-invariant representative. 
The values of the invariants on the curves bw;,9 and cw;,o are obtained 


applying (iii). 


In the case of the curves w;,,, we start from w1,0 and transform 
it into the curve 7 (see fig. 34) so that y = a#(@ where a and G are 
two heart curves, both of index 1 but with opposite coorientations, so 
that they have Maslov indices 2 and —2, respectively. ( a@ = w 1,2 and 
Cos = W1,-2). 


OR BSS BOD 


C= iy SO = ca # @ 
77 0 4 8 8 4 8 0 
76 CU 0 0 0 0 0 0 
p' 0 0 0 0 0 0 0 
pt 0 0 0 0 0 0 0 
I 0 0 0 0 0 2 2B 
M 0 0 0 0 0 2 2 


Figure 34. The curves a and ca 
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We obtain: 
Ve Pease 
ea 
3) pl(a) +p! (ca) = 0 
4) pr(a)+p*(ca) =0 
5) Al(a)+Al(ca) =2 
6) A(a) +A! (ca) = 2 


From (1) (or from (2)) applying (iii) one has 
IG) =) 
but a satisfies abca = a, hence again from (iii) 
f* (abea) = f(a) = fT (a). 


Therefore f* and f~ are both vanishing on all the four front classes of 
the heart curve. 

From (iii) p'! (ca) = —p! (a), thus eqq.(3) and (4) do not give any 
information. 

The values of the invariants \' and \! do not change under the 
involution c. Hence equations (4) and (5) give: 


We have still to find the value of the invariants p! and p* on the 
basic curve W1,2. 
Consider the two curves 6 and ¢ shown in fig. 35 


One has: 


6 =a # Woo # aa, 
C =a # a F# CH. 


We can calculate all their invariants since 
jC) ago.) ee Gaya) ae lator) = 0) G=)\) 


while the values of p"(¢) are found transforming 6 into ¢. (See fig. 36) 
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ry) a Wo.0 aa 


2 


¢ (a4 a cWo ,0 


Figure 35. The two curves 6 and ¢ 


§ > yy SO wn oS Bp 6 = a FH A HF cw 
[ee 4 0 0 0 0 0 0 
Pe 0 0 2 2 0 0 2 
pi 0 2 D 2 2 1 i 0 
p- 0 0 0 2 2 i i 0 
io? 2 2 D 2 d 1 0 
+ 2 2 2 2 2 1 1 0 


Figure 36. A generic path from 6 to ¢ 
Hence we obtain from 
ig 2 ep a) pea ia) 
that p'(a) = p!(a) = 1. 


Having the values of the basic invariants on the basic curve @ = W1,2 
we find, by connected sum, the values on the basic fronts 1... 


Remark 3.7. Every curves 1,, has a representative which is invariant 
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under Rabe). 


The values of the RO-dependent invariants on the curves w,,,, are 
finally calculated as sums of the values of these invariants on the curves 
Wy 0 and Wu 

To finish the proof of the theorem, we have to verify the condition 
(ii). As in the case of RO-independence, it suffices to verify it on the 
basic curves. 

We can repeat step by step the proof of Theorem 2.1, taking this 
time into account all the orientations. This is rather long. Therefore 
first. of all we observe the two following facts, whose proof is in §3.6: i) 
For a generic oriented and cooriented front in the oriented plane whose 
L-immersion belongs to 9;,,, the invariant A! (A!) (defined by its jump 
and its values on the basic curves w;,, shown in table 7) is equal to 
the number of the cusp points of the front of type up (of type down 
respectively); ii) the invariants ft (f—) (defined by its jump and its 
values on the basic curves shown in table 7) is equal to twice to the 
number of the double points of the front such that the two crossing 
segments are both positive (both negative respectively). 

The additivity of the invariants \', A, f+ and f~ under the con- 
nected summation follows immediately from this observation. 

To prove the additivity of the invariants p' and p! we proceed as 
in the proof of Theorem 2.1 (see §2.4). 

The events of cusp crossing which take place in the transformation 
of a connected sum of basic curves into a basic curve, occur in the sets 
of perestroikas called P, and Pz. The variations of p' and p* at P, are 
both vanishing. The variations at P2 are both equal to -2, in the case 
shown in fig. 37. (The variations of p' and p! in the case of different 
orientations is found applying the lemma 3.3). 


We consider now the different cases of connected summation of 
basic curves depending on the positions of the gluing points on the 
curves. If the gluing point on a curve is of type (3) or (4) (see fig. 38), 
we can always transform the curve before the gluing so that this point, 
remaining always generic and at the boundary of the curve, becomes 
of type (1). We have only to transport cusps points beyond loops. It 
is achieved by the perestroikas P,, which do not change the values of 
p' and pt. 

We remark that in the case of a point of type (3’) (i.e., when the 
number of cusp points to be transported is odd), the resulting basic 
front (containing the gluing point at the position (1)) is obtained from 
the original front by the abc involution. This involution does not change 
the value of the invariants p! and p! (see Remark 3.4). 
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PPX QR DA 
Vi. <— QQ EQ 
= =} 


es 


a 


Figure 37. The P; and P, perestroikas 


{ 
=) 


(3) /\ (3) 


Figure 38. Different places of the gluing points 


The perestroika P2 (the only one changing the value of p!!) occurs 
only when the gluing points are of type (1)-(2) (see fig. 24). In the 
case of fig. 24 the perestroikas P2 are exactly of the type considered in 
fig. 37. In the transformation of the connected sum of the basic curves: 


(ii Ot 3) 
Wi pAPW—j,v 


the perestroika P, occurs exactly k = y/2 times. Hence the total 
variation of p’ and p! are both equal to —2k. This is in agreement 
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with the values of p! and p! on the basic curves (see Remark 3.4): 


Pp" (Wi2m) + P"(W_z,2k) =k-—m 


DO (Oi avon) 22 Ail = 8 


where 7) = (f, |). Hence the invariants p' and p! are additive under the 
connected summation of the basic fronts of type (1)-(2). 

In all the other cases of transformation of the connected sum of 
two basic fronts into a basic front, when the perestroika P2 does not 
enter, one verify easily that the value of the invariant p' and p! on the 
resulting basic curve is equal to the sum of the values of these invariants 
on the two original basic curves. 

The proof of the existence of the additive invariants is thus com- 
pleted. 


II) Uniqueness of the additive invariants. 


Lemma 3.6. An additive invariant of oriented and coortented fronts in 
the oriented plane is unambiguously defined by its jumps, provided that 
it is invariant under the reversal of the coorientation of the front. If at 
is not required to be invariant under the reversal of the coorientation 
of the front, it is defined by its jumps up to the addition of a multiple 
of the Maslov index. 


Proof. The difference of two invariants with equal jumps is a function 
locally constant. Hence it may be considered as a function on the set 
of the components 1;,,, of the space of the legendrian immersions. 

The connected sum defines two operations on this set. Indeed, 
there are four possible types of bridges connecting two oriented and 
cooriented fronts, one in the left half-plane, the other in the right half- 
plane (fig. 39). 


ae ea 
FR FR IN IX 


Figure 39. The four types of bridges connecting two fronts 
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Using the elementary perestroikas one is able to prepare on the left 


and on the right curves two pieces, ready for the joining with any of 
the four bridges (fig. 40). 


| 
y= Pee 
etl 


Figure 40. Preparation of the pieces ready to the joining 


Therefore one can add two curves of any two classes Qn aude 
by any of the four bridges. 

The Maslov index is additive under any of the four types of the 
connected summation. The index of the sum is iy +%24+ 1 for the first 
two bridges (differing only by the coorientation) and i; +%2 —1 for the 
other pair of bridges (obtained from the first pair by the orientation 
reversal). Hence the connected summation defines two operations on 
the set of connected components Z? = {i,k = Nay 2 


Ag : (i, k1), (iz, kg) > (a + 2 +1, hr + fa) 
eee: (41, k1), (¢2, ka) 7 (i4 +%2—-1,k, + k) 


Both operations are commutative as operations on Le 
The additivity of the difference of two additive invariants with equal 
jumps for the A+ connected sum means that this function on the affine 


72 F. Aicardi 


Z? is linear with respect to the abelian group structure in Z*, whose 
null element is (¢ = —1,k = 0). All such functions have the form 
f(t,k) = a(i+1) + 5k. Similarly, the additivity with respect to the 
second kind of connected sum implies that f(7,k) = c(i — 1) + dk. 
Hence f (i,k) = ek, which proves the lemma. 


Corollary 3.3. The only RO-independent local additive invariants are 
the linear combinations of (J*,J—, Sp, h, ). 


Proof. Indeed, the linear combinations of the basic RO-independent 
jumps are the only possibilities for the RO-independent jumps (see the 
proof of Lemma 2.2) and the jumps define unambiguously the invariants 
by the above lemma. 


3.6. Explicit formulas for some basic local invariant 


Explicit formulas for the local invariants of the immersions of a 
circle in the plane and for the RO-independent invariants of the planar 
fronts have been found [5,6,7|. There are two simple formulas for the 
invariants ft, f—, A' and A! of a generic front ¥. 

Let n' and n! be the number of cusp points of y of type up and of 
type down respectively. 


Proposition 3.1. 


Proof. This is true when ¥ is the basic curve w;,, (u = 2k) having 
k cusp points of type up and k cusp points of type down, and having 
A! = dt =k. At the birth of two cusp points, they are both of type 
up or both of type down at the corresponding increment by 2 of A! or 
of At. The character up or down of a cusp does not change along any 
path in 0; ,.. 


Let n* (n—) be the number of double points of y such that both 
intersecting branches are positive (negative). 


Proposition 3.2. 


= a 


Proof. For the basic curve y = w;,, (see table 7), the number of 
such double points is nt = i1+1 while ft = 24@+1), f- = 0. The 
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numbers of double points of ay, by and cy are equal to n™ =7+1 while 
je 0,3 — 20 1). 

The number nt increases by 2 at the positive crossing of the local 
stratum K eS , and at the negative crossing of the local stratum Kp, , 
and remains unchanged at the crossing of the other local strata of self- 
tangencies. At the strata of cusp crossings, n* increases by 1 when 
Ar2+ = 2 and reamins unchanged in the other cases. Indeed at the 
cusp crossing where the smooth segment crossing the cusp is positive, 
Ar2+ = 2 and Ant = 1 (since one of the branches into which the cusp 
point divides the cusped segment is always positive). If the smooth 
segment is negative, Ar?+ = An* = 0. Moreover, n* increases by 1 
while f+ increases by 2 at the positive crossings of the local strata AWe 
and (eke Hence f+ = 2n+ for any planar front. Similar arguments hold 
for the invariant n-. 


Remark 3.8. From the preceding observations it follows that the 
values of the invariants ft, f—~, A’ and A! cannot be negative. 


Remark 3.9. From the equality (see Theorem 2.1 and Corollary 3.1): 
a ee ee 
one obtains the result of M. Polyak ([8]): 
UE ely = i il —'e 


where n4 is the number of double points of the front where the two 
intersecting branches are both positive or both negative, n_ is the 
number of double points where the two intersecting branches are one 
positive and the other negative and 2c is the number of the cusp points. 


Indeed, ny =3(ft +f"), nyetn-=n and c= 3A. 


3.7 Other types of invariance 


The local basic additive invariants are functions constant on the front 
classes (on the orbits of the generic legendrian immersions of a circle 
to the space of the cooriented contact elements of the plane) under the 
action of the group of the orientation preserving diffeomorphisms of the 
source circle and of the target plane. 

Some of the invariants we find are also invariant under the action of 
larger groups of diffeomorfisms of the plane and of the circle which are 
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not necessarily preserving all the orientations. We obtain the following 
facts from Theorems 2.1 and 3.1. 


Corollary 3.4. The basic local additive invariants of the front classes 
of legendrian immersions of the circle in the space of the cooriented 
contact elements of the plane up to diffeomorphisms preserving only 
the orientation of the plane, only the orientation of the circle, or non 
preserving these orientations at all, are shown in the following table 


Rand 5) i gee = So ey ee pe ee ee 
1B Hy oe), SD, eo felon (Duce cee 
oe a I ge ey ltr), (Lee 12) 
- TE SSD as =), (Al +2") 


Remark. In the last three cases only the absolute values of the index 
is invariant. 


Proof. Consider the symmetry types of the reduced elementary 
jumps which constitute the basic RO-dependent invariants (see §3.3). 
For example, an invariant of a front class defined up to diffeomorphisms 
preserving only the orientation of the plane has to be invariant with re- 
spect to the 6 involution (which reverses the orientation of the preimage 
circle), and so on. Of course the five RO-independent basic invariants 
are still invariants in the four cases. 


Consider now a similar problem, when the front is defined as the 
projection to the plane of a legendrian immersion of a circle in the 
space of the noncooriented contact elements of the plane, with the same 
contact structure as in the cooriented case. 

All the codimension 1 and 2 strata of the discriminant can be de- 
fined in this case, where the normal vector of the front is absent. In 
this case the answer depends on the parity of the the number of cusp 
points. This parity is invariant under all the perestroikas, the birth and 
the death of cusp points occurring in pairs. 

We study only the case of an even number of cusp points. The 
index, i.e., the number of complete turns of the normal vector of the 
front, is thus an integer number. If the orientation of the preimage 
circle or that of the plane is not preserved, then only the absolute 
value of the index is invariant. 

In terms of the front, the definition of the Maslov index as the 
difference betweeen the numbers of positive and negative cusp points 
is no longer possible. 
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The front of a legendrian immersion of a circle into the space of 
the noncooriented local elements of the plane with an even number of 
cusp points has two coorientations. The Maslov index of these two 
cooriented fronts are opposite. The absolute value of each of them is a 
well-defined (non additive) invariant of the noncooriented front. 


Corollary 3.5. The space of the local additive invariants of the non- 
cooriented fronts with an even number of cusp points has dimension 6 
in the case where the plane and the front are oriented. The generators 
of this space are (J*+,J~, Sp, ft + f-,A', A‘). 


The spaces of the local additive invariants of the noncooriented 
fronts with an even number of cusps which are invariant under the 
reversal i) of the orientation of the plane; 11) of the orientation of the 
front; iu) of both orientations, coincide and have dimension 5. Their 
generators are (J*,J~,Sp, ft + f7,Al +t). 


Proof. Acting by the coorientation reversal involution c on one of the 
two cooriented fronts 71 covering the given noncooriented front y, one 
obtains the other cooriented covering front 72. The arithmetical mean 
of the values of a local additive invariant on the two cooriented fronts 71 
and 42 = C¥2 is a local additive invariant of the original noncooriented 
front. In fact this construction provides all the local additive invariants 
of the noncooriented fronts. 
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Crofton Densities, Symplectic Geometry 
and Hilbert’s Fourth Problem 


J. C. Alvarez ILM. Gelfand M. Smirnov 


Abstract 


We study the relation between the integral geometric and the 
symplectic construction of Desarguesian metrics on R” and show that 
these constructions characterize all Desarguesian Finsler metrics. 


1. Introduction 


A Desarguesian metric on an open convex subset O Cc R” is defined by a 


distance function 
d:0xO—R 


with the usual properties: 


D1. d(z,y) >0 

D2. d(iz,y)=O0Sa2r=y 

D3. d(x, y) = d(y, 2) 

D4. d(x, z) < d(z,y) + d(y, z). 


Together with the Desargueness condition: 
D5. If z,y, and z are collinear points and y is between x and z, then 


d(x, z) = d(x,y) + d(y, 2) - 


If y : [a,b] — O is a continuous curve we define its arclength with 
respect to a metric d by the following formula: 


k-1 


I(y) := sup{) | d(y(ti), y(tit)) | 
i=0 
a=ty <-::<t, = bis a partition of (a, bl} . 


Clearly, a metric d on O is Desarguesian if and only if the shortest 
curve between two points is the line segment joining them. The problem 
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of finding all geometries for which the line segment is the shortest curve is 
known as Hilbert’s fourth problem. 


In this paper we present some constructions of Desarguesian metrics 
and study their degree of generality. More specifically, we give variants 
of the integral geometric construction of Busemann, the symplectic con- 
struction of Alvarez, show the beautiful relation between them, and prove 
that any Desarguesian Finsler metric may be obtained by either of these 
constructions. 


The present paper is based on the work of the first author on the 
symplectic geometry of spaces of geodesics (see [Al] and [A2]) and the 
work on Crofton densities ({GS]) by the second and third authors. The 
relation between symplectic and integral geometry resulted in a clear and 
simple understanding of both the symplectic and the integral geometric 
constructions of Desarguesian metrics. 


For the authors, a great part of the beauty of our approach to Hilbert’s 
fourth problem is that it relates a number of concepts which are 2- 
dimensional in nature: 


e A Desarguesian metric on R” induces on every 2-dimensional affine 
subspace a Desarguesian metric. 


e A subset of R” is convex if and only if all its 2-dimensional sections are 
convex. A function on R” defines a norm if and only if its restriction 
to every 2-dimensional subspace defines a norm. 


e A symplectic form, as a 2-form, ultimately depends only on its re- 
strictions to 2-dimensional submanifolds. 


A word on the organization of the paper: in Section 2 we present 
some constructions of Desarguesian metrics. Section 3 relates Hilbert’s 
fourth problem to the calculus of variations by discussing Finsler metrics 
and variational problems whose extremals are lines. In section 4 we show 
that the integral geometric construction presented in 2.4. characterizes all 
Desarguesian Finsler metrics. Finally, in Section 5 we relate the integral 
geometric construction and the symplectic construction proving that the 
symplectic construction also characterizes all Desarguesian Finsler metrics. 
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2. Constructions 


In this section we review some constructions of Desarguesian metrics on R”. 
For the most part this constructions are well known, but we have modified 
the integral geometric construction of Busemann and the symplectic con- 
struction presented in [A]. in order to underline the relation between the 
two. 

2.1 Minkowskian Geometries 


The simplest construction of Desarguesian metrics is due to Minkowski. In 
modern terminology, it amounts to giving a Banach norm on R”. 


Given a bounded convex set C C R” which is centrally symmetric with 
respect to the origin we define the distance d by the following procedure: 


Draw the line segment joining a point x with 0 and denote the point 
of intersection with the boundary of C by zo. We now define d(z,0) by 


the equality d(x, 0)0x0 = Oz. If x and y are two points in R” we define 
d(x,y) = d(x — y,0). 

The space R” with such a metric structure is called a Minkowskian 
space. It is easy to see that a Minkowskian space is Desarguesian. 


Note that Minkowskian geometries are characterized by their transla- 
tion invariance. 


2.2 Hilbert Geometries 


In [H] Hilbert noted that the Cayley-Klein model of hyperbolic geometry 
could be modified to yield Desarguesian metrics on open bounded convex 
subset of R”. 


Let O C R®” be an open bounded convex set, and let and y be 
two points in O. Draw the line defined by x and y and mark the two 
intersection points of the line with the boundary of O. Denote by B the 
intersection point such that y belongs to the segment xB, and denote the 
other intersection point by A. Now define 


isl ly = Bl) 
ER eg ae de 
(2) =item 


where || - || denotes the Euclidean norm. 


The multiplicativity of the cross ratio immediately implies that the De- 
sargueness property is satisified. Unlike Minkowskian geometries, Hilbert 
geometries have no obvious symmetries. However, like Minkowskian ge- 
ometries, the metric is not Riemannian unless the convex set used to define 
it is an ellipsoid. In this case we have the Cayley-Klein model of hyperbolic 


geometry. 
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2.3 2-D Case 


It is to be noted that, roughly speaking, the construction of Minkowskian 
or Hilbert geometries in n-dimensional space depends on a function of n- 
1 variables (the choice of a convex hypersurface). We shall now give a 
geometric construction of Desarguesian metrics on R? that depends on a 
function of 2 variables. 


Consider an area form on the cylinder {(z,y,z) € R® : 2? + y? = 1} 
which is invariant under the antipodal map. We associate to this area form 
a metric on R? as follows: 


Identify R? with RP? . RP! or, more precisely, with the set of planes 
in R? which pass through the origin but do not contain the z-axis. If x and 
y are two such planes we consider their intersection with the cylinder. This 
intersection consists of two ellipses which divide the cylinder into four con- 
nected components, two bounded and two unbounded. As a consequence 
of the invariance under the antipodal map, the bounded components have 
equal areas. We denote the common area by a(x, y) and define the distance 
between x and y by 


dx y= 50x, y). 


If x,y and z are three collinear points in R?, then the planes they 
represent intersect at a line. In the cylinder we have three ellipses that 
intersect in two antipodal points. It is easy to see that if y is between x 
and z, then a(x,z) = a(x, y) + a(y,z). 


The reader may enjoy checking that if the area form on the cylinder is 
the one induced by its embedding into R°, then the resulting metric on the 
plane is the Euclidean metric. Note also that if the area form is independent 
of the z-coordinate on the cylinder the resulting metric is Minkowskian. 


2.4 Integral Geometric Construction 


Let H denote the set of all affine hyperplanes in R”. The set H inherits 
a manifold structure from its natural identification with the quotient of 
S"-1 x R by the involution (€,p) ++ (—€,—p). The manifold S"~1 x R is 
itself identified with the set H of all oriented affine hyperplanes in R”. 


Let o be a smooth signed measure on H. Given two points z,y € R” 
we denote by Hz, the set of hyperplanes which intersect the segment ry. 
Let us define d(x, y) := o(Hzy). 


We would like to find conditions on o which assure that d defines a 
metric on R”. It is easy to see that such a metric would necessarily be 
Desarguesian. 
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Quasi-positive measures 


Definition 1 A smooth signed measure o on H is said to be quasi-positive 
if it satisfies the following condition: If U is a Borel subset of the set of 
lines lying on a 2-dimensional affine subspace P C R” and U is the set of 
all hyperplanes that contain a line in U, then eeu) = o(U) > 0; 


Theorem 1 Leto be a smooth quasi-positive measure on H, the function 
d:R” x R” — (0,00) 
defined by d(x, y) := 0(Hzy) defines a Desarguesian metric on R”. 


The Desarguesian metrics obtained by this construction will be called 
o-metrics. 


Let us elaborate somewhat on the quasi-positivity condition: 


Let P C R® be a 2-dimensional affine subspace, and let Hp C H 
denote the set of hyperplanes intersecting P on a line. If L(P) denotes the 
set of lines lying on P, then we have a natural projection 


tp:Hp — L(P). 


The push forward of the measure a under the map 7p is what we 
have called op. For every 2-dimensional affine subspace P a quasi-positive 
measure induces an area form on L(P). This reduces the construction of 
Desarguesian metrics to the 9-dimensional construction. 


When the measure o is invariant under the action of the translation 
group on H it can be identified with a measure on the projective space 
RP"-!. In this case the o-metric is Minkowskian and is, therefore, defined 
by a convex subset of R”. The explicit relation between quasi-positivity 
and convexity will be presented in section An: 


2.5 Symplectic Construction 
Let L(R”) denote the set of all oriented lines in R". The set L(R”) inherits 


a manifold structure from its natural identification with the tangent bundle 
of the (n-1)-dimensional sphere, gtr: 


An oriented line is specified uniquely by its direction vector and the foot 
vector defined by the point on the line closest to 0. The direction vector 
lies in the unit sphere and the foot vector is orthogonal to it. Together they 
define a point in T'S”7?. 


Let w be a differential 2-form on L(R”). Let z,y be two points in R” 
and choose a 9-dimensional affine subspace P C R” containing x and y. 
Denote by L(P) the set of all oriented lines lying on P and let 


ip: L(P) — L(R") 
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be the natural inclusion. 


Set 
d(z, y; P) =| jip*w|, 
L(P)zy 


where L(P)zy € L(P) is the set of all oriented lines on P intersecting the 
segment ry. 


We would like to find conditions on w which assure that the quantity 
d(x, y; P) is independent of P and defines a metric on R”. It is easy to see 
that such metric would necessarily be Desarguesian. 


Admissible Symplectic Forms. Before defining admissible symplectic 
forms let is quickly review the definitions of symplectic form and Lagrangian 
submanifold. 


Definition 2 A differential 2-form w on a smooth manifold M 1s said to 
be symplectic if it is closed and at every point m € M the bilinear form 


Wm: limM xT,M—R 
is nondegenerate. 


Definition 3 Let M be a smooth manifold and let w be a symplectic form 
on M. A smooth submanifoldT C M is said to be Lagrangian if w restricted 
to T is identically zero, and the dimension of T is half that of M. 


Let [, c L(R”) denote the set of all oriented lines passing through a 
point x € R”. Clearly the I, are smooth embedded spheres in L(R”). 


Definition 4 A symplectic form w on L(R”) is said to be admissible if 
for each point c € R” the submanifold [, is Lagrangian, and if a*w = 
—w, where a: L(R”) — L(R”) is the involution defined by changing the 
orientation of the lines. 


Theorem 2 Let w be an admissible symplectic form on L(R"). The func- 


tion 
(x,y; P) = rp jip* | 
L(P)cy 
is independent of P and defines a Desarguesian metric on R”. 


Distance functions on R” obtained by this construction will be called 
w-metrics. 


Note that both the symplectic construction and the integral geometric 
construction, as presented in the previous section, reduce the problem of 


Crofton Densities 83 


finding Desarguesian metrics on R” to the problem of finding a compatible 
choice of positive measures on the family of submanifolds L(P) c L(R") 
parametrized 2-dimensional affine subspaces of R”. For the integral geo- 
metric construction the measures are the pushforwards ap, for the symplec- 
tic construction they are the absolute values of the pullback forms ip*w. 


3. Desarguesian Finsler Metrics 


The constructions of Desarguesian metrics presented in the previous sec- 
tions all yield continuous metrics (i.e., the topology induced on R” by 
these metrics is the standard). The smoothness requirements on the quasi- 
positive measures and admissible symplectic forms further restricts the class 
of metrics that can be obtained by these constructions. The main restric- 
tion is that these metrics be defined “infinitesimally.” For example, the 
Euclidean length of a curve -y on R” may be defined by the integral 


sta) = f Ieee 
Mi 


The Euclidean distance between two points z,y € R” is now recovered 
through the formula 


d(x, y) := inf{S(y) : 7 jos z and y} . 


One may now try to abstract the properties of the integrand l(t) || that 
garantee that d defines a metric on R” and then one may try to classify all 
Desarguesian metrics of this form. In this section we study these metrics 
and give the precise statements of our main results. 


3.1 Finsler Metrics and Even 1-Densities 


Let y : R™ x (R” \ 0) — R be a smooth function. If 7 is a smooth 
parametrized curve on R” we define its action by the integral 


Solr) = f eleaat 
ei 


To make the integral invariant under reparametrizations of yy we require 
that the function ¢ satisfy 


p(x, Ax) = |Aly(a, £) , 


for any nonzero real number 4. Such a function will be called an even 
1-density. 


We propose constructing metrics of the form 


d(x,y) := inf {S (7) : 7 joins x and y} , 
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where y is an even 1-density. It is well known that the following are suffi- 
cient conditions on an 1-density y for d to define a metric on R”: 


e y(z,z) > 0, for any « #0 
e The matrix of second derivatives of the squared function y? with 


respect to the the z’s 
O2p? 
ee ij 


is positive definite. 


The metric d obtained from a 1-density y satisfying the above condi- 
tions is said to be a Finsler metric. We will also call the 1-density y a 
Finsler metric. 


We now state our main results. 


Theorem 3 A Finsler metric on R” is Desarguesian if and only if it is a 
o-metric. 


Theorem 4 A Finsler metric on R” is Desarguesian if and only if it is 
an w-metric. 


3.2 Hamel’s Theorem 


When an even 1-density y is not Finsler we may no longer speak of its 
geodesics as the curves which locally minimize length, but we may speak of 
its geodesics as extremals. These extremals are the solutions of the system 
of Euler-Lagrange differential equations: 


d ( 0 Op : 
= (3) -# = ) ele 


Hamel, a student of Hilbert, was the first to study the Euler-Lagrange 
equations of a 1-density whose extremals are straight lines. We state his 
result with some modifications. 


Theorem 5 An even 1-density y satisfies the equations 


oS Py 
Ox, 0%; _ Ox;0z% lena] = ean soe aV) 


if and only if straight lines are geodesics. 
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The similarity between this system of equations and the equations for 
a closed 1-form on R” was noted and clarified in [GS]. In the k-dimensional 
case [GS] similar equations can be written and these equations similar to 
equations for closed differential k-forms. Other analogues of differential 


forms and De Rham complex were studied by M. Baranov and A.S.Shvarts 
[BS]. 


We may now pose the problem of characterizing all even 1-densities on 
R” whose extremals are straight lines. This problem may be taken as the 
first step in the integral geometric solution of Hilbert’s fourth problem. 


4. Crofton Densities 


In this section we present the main ideas that go into the proof of theorem oO 
We start by giving an integral geometric characterization of even 1-densities 
whose extremals are straight lines. 


Most of what follows has been taken from [GS]. 


If 7 is a curve on R” and ¢ € H is a hyperplane, we define Crof,(¢) 
to be the number of points in yM¢. If 7 is a regular curve, the set of 
hyperplanes for which Crof,,(¢ ) is infinite has measure zero. 


Definition 5 (GS) An even 1-density p in R” is called a Crofton 1- 
density if there exists a signed measure p(¢)d¢ on H such that for every 
smooth curve y in R” 


[oe [ cret(ootoas . 
5 H 
The measure p is called the dual measure of the Crofton density p. 


Theorem 6 (GS) A smooth even 1-density p ts a Crofton density if and 
only if its extremals are straight lines. 


Theorem 3 follows from the preceding theorem and the following char- 
acterization of those Crofton densities which are Finsler: 


Theorem 7 A Crofton density is Finsler if and only if its dual measure 1s 
quasi-positive. 


4.1 Sketch of the Proof of Theorem 6 


In what follows it will be advantageous, from a notational point of view, to 
work with the set of oriented hyperplanes H. The reason is that H is an 
orientable manifold and we may work with volume forms instead of smooth 
measures. Moreover, using the natural identification of H with R x S”"}, 
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we may write any differential n-form 7 as v(p, €)dpAQ, where v is a function 
on R x S"~!. Differential n-forms which are even, or odd, with respect to 
the involution (p,€) ++ (—p,—€) induce smooth signed measures on H. 
We will only consider such forms. An n-form 7 on H will be said to be 
quasi-positive if it induces a quasi-positive measure on H. 


Our point of departure is an interesting and classical integral represen- 
tation of smooth homogeneous functions on R” \ {0}. 


Theorem 8 Let y: R” \ {0} — R be a smooth function homogeneous of 
degree one. There exists a unique smooth even function ps on S"-1 such 
that 


(0) = Egy a6 HOD 


From this theorem we obtain an integral representation of even 1- 
densities. 


Proposition 1 If p(z,v) is an even 1-density on R”, then there exists a 
unique smooth function p(z,&) on R” x S"—! which is even in €, and such 
that 


lay) = fogs IE elH(@, 6). 


In the context of the integral representation of even 1-densitiesm, the 
representation of Crofton densities is as follows: 


Proposition 2 (GS) A smooth even 1-density y is a Crofton density if 
and only if there exists a smooth even function v(p,£) on R x S"-1 such 
that 


elc,v)= fle vlv-2,8), 
éeEsr-1 
where Q. is the standard volume form on the untt sphere in R”. 


Finally, the proof of theorem 6 follows from Hamel’s theorem and the 
following technical result: 


Theorem 9 The 1-density yp(z,v) = ih 


ecg! |f- vju(z,€)Q satisfies the 


equations 


Op Pp 
= —, k,j =... 
Ox,0v;  OxjOv,’ paar 0 


if and only if there exists a smooth function 


vy. Rx S"-! — RR 


such that p(n ey (E - 2, 6) 
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4.2 Proof of theorem 7 


As in the proof of Theorem 6, we start by studying the representation 
of even 1-densities which are invariant under translations. In this case a 
Finsler metric is just a Banach norm with certain smoothness requirements. 
The following is the precise definition of the norms we will work with: 


Definition 6 A smooth function y : R” \ {0} — R homogeneous of degree 
one is said to be a norm function if 


e y(v) > 0, for anyv £0 


e The matrix of second derivatives of the squared function y? 
0? —p? 
tes ij 


The strong convexity conditions required by the the positive 
definiteness of the Hessian assure that not only the norm, but also the 
dual norm is smooth. 


is positive definite. 


The problem of characterizing the functions p: S"~! — R such that 
pw) = fo, - vlale)e. 
©) f guile 


is a norm function is known as Funk’s problem. The notion of quasi- 
positivity allows us to give the following solution : 


Theorem 10 Let »: S"—1 — R be a smooth even function. The function 


~i= if gat IE - vlu(E)2 


is a norm function if and only if w(€)dp AQ. ts a quasi-positive top order 
form on H. 


This theorem follows simply from the remark that a homogeneous func- 
tion of degree one is a norm function if and only if the restriction to any 
2-dimensional vector subspace is a norm function, and the following result: 


Proposition 3 Let ¢ : R? — R be a smooth function homogeneous of 
degree one, and let 4: S' + R be a smooth even function such that 


20 
$(v1, v2) = | |cos(A)v; + sin(O)v2|u(8)de . 


The function ¢ is a norm function if and only if p> 0. 
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The characterization of those even 1-densities which are Finsler is now 
straightforward. 


Proposition 4 An even 1-density y(x,v) is a Finsler metric if and only 
of 
ete = fo vlan), 
(z,v) aero! 


where for each point x € R” the form p(z,£)dp \Q is a quasi-positive 
n-form on H. 


Theorem 7 follows directly form this result and the characterization of 
Crofton densities given by proposition 2. 


5. Integral Geometric Construction 
of Admissible Symplectic Forms 


In this section we show how to construct admissible symplectic forms on 
L(R”) from quasi-positive top order forms on H. 


Let H be the space of oriented hyperplanes on R” and let 7 be a top 
order form on H. The form 7 induces a 2-form R(7) on the space of oriented 
lines by the following procedure: 


Let A= {(l,h) € L(R”) x H:1 Ch} and let 
i(k") A B 
be the natural projections. 


If 1 € L(R®) is an oriented line and wi, w2 € T,L(R”) are two tangent 
vectors over I, we choose at each hyperplane h D I two tangent vectors 
1, Wa € ThH such that w; and w; have a common preimage in Ty; ,p) A. 

_ The contracted form 2.5, rnw27h vdoes not depend on the choice of #1, we. 
We define 


R(n)i(wi, we) = || bby Aba Mh + 
hdl 
Roughly speaking, the 2-form R(7n) at the line / is the integral of the 
form 7 over the codimension 2 submanifold {h € H: h D /}. 


It is easy to see that the form #(7) is necessarily closed. 


Theorem 11 A symplectic form w in L(R") is admissible if and only if 
there exists a quasi-positive n-form n on H. such that w = R(n). 


We may now use this characterization of admissible symplectic forms 
to prove theorem 4. Indeed, it is easy to see that if 7 is a quasi-positive n- 
form on H which induces a quasi-positive measure o on H and an admissible 
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symplectic form w := R(7) on L(R”), then the pushforward measure op 
and the measure |ip*w| coincide on each 2-dimensional affine subspace P € 
R”. It then follows that the o-metric defined by o coincides with the 
w-metric defined by w = R(n). 


6. Crofton densities and Desarguesian Finsler metrics 
as even differentials of functions 
and Hilbert transforms of closed 1-forms 


In this section we demonstrate that there is another way to construct 
1-densities whose extremals are straight lines. 

We recall that an even 1-density is a function y(z, v) of a point x € R” 
and a tangent vector v at the point x such that y(z, Av) = |Aly(z, v) for 
every ACR. 

An odd 1-density is a function 6(z, v) of a point zc € R” and a tangent 
vector v such that 6(z, Av) = AO(z, v) for every \ € R. Differential 1-forms 
give a specific example of odd 1-densities, they are linear in v. 


Definition 7 The 1-density (Hy)(z, v) 
ive t 
(Hyp) (2, v) = P.V.— / ese = 


+00 
i p(x + vt, v) — p(x — vt, v) a 
3 t 
0 


is called the Hilbert transform of a 1-density y. 


Here we suppose that y satisfy growth conditions for example it is rapidly 
decreasing. This means that y(z — vt, v) is a rapidly decreasing function of 
t for any fixed v and x). So its Hilbert transform exists. 


Theorem 12 The Hilbert transform of an even 1-density is an odd 1- 
density. The Hilbert transform of an odd 1-density is an even 1-density. 
The Hilbert transform of a closed 1-form with rapidly decreasing coefficients 
is the Crofton 1-density. The Hilbert transform of a rapidly decreasing 
Crofton 1-density yp is a closed 1-form. 


Let f be a function in R”. Its differential df depends on a point x from 
R” and a tangent vector dx = v from T,;R” can be written as 


+00 


(ayte,») = f sie vat 


—o0o 
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Definition 8 The nonlocal (or even) differential of a function f is 
yi ran + ut) + f( i) 2 F(z) 
4 0 Uv G5 > OG |) oR 
(d° f)(2,v) = / J, fle ~ vi)dt= / tee), 
mes a 


We suppose that f satisfies some growth conditions so that this integral 
converges at infinity. 


Let f(x) be a rapidly decreasing function in the Schwartz space S(R”). 
Suppose that f is the Radon transform of a function F(a): 


f (Gis — 


+00 
= / F(a, se pedi = (Ota cate Qn—1En—-1)) day iho a, 
Let 
+00 1 
olan) =(@Ale,) = fe f@-viae 


be an even differential of the function F’. Then 
(1) y(z, v) is a Crofton density in R”. 
(2) Let us denote by D2 F(ai1,...,@n) , an “even partial derivative” of F 
i 
DG en) = f pPloa.-10n— tat 
Then the D°F is the dual function for y: p(a,--.-,Qn) = 
DG. c,.). 


Radon transf. 


ey 


UL Even Radon tr, d o i 
| 
~ 


Theorem 13 Suppose f is a rapidly decreasing function, w = df ts its 
differential and yp = —d°f is its nonlocal differential (which is the Crofton 


1-density). Then (Hy)(z,v) = w(x,v) and (Hw)(z,v) = y(z,v). So the 
Hilbert transform maps even differentials into odd ones and vice versa. 
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We can summarize properties of Crofton 1-densities on the following 
diagram: 


F Radon transf. a f Radon transf. F 
DD: ‘ | 
) d d D 
n an 
m Even Radon tr. d a Hilben a. df Od Radon tr. n 
@w 
Crofton 1-density Closed 1-form 
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Projective convex curves 


S. Anisov 


Abstract 


We deal with convex curves and surfaces in real projective space. Ques- 
tions concerning the affinity of convex curves, the flattening points and the 
loss of convexity are regarded. Finally, we prove that there are no convex 
surfaces except curves and hypersurfaces. 


1. Main Definitions 


Definition. (See [1]). A smooth embedding I: S' — RP” is said to 

be a convex curve if every hyperplane intersects (vy) no more than n 

times according to multiplicities. The parameter y on S 1 is defined to be 
mod 27. 


Since n-multiple crossings are allowed (but not (n+ 1)-multiple ones), 
we must suppose that the map I’ is smooth of class C”. 

Suppose also that 0 4 a (y) (i.e., D,I). Otherwise a transversal 
intersection can incidentally be regarded as a multiple one. 


Example. I'(y) = (1:¢:t?:...:t"t'), where t = tan £, is a smooth 
closed convex curve in RP”; I'(y) = (sinw : cos: sin3p : cos3p:...: 
sin(2n+1)p : cos(2n+ 1)w), where p = €, is a smooth closed convex curve 
in P2"*", 


Let Ai,...,Ax be different points of P(y) and m1, ... ,™x integers 
such that m, +...+m, =n. Then there exists a hyperplane that crosses 
I at A;,i=1,...,k with multiplicities m;. If the curve T is convex, then 


such a hyperplane is unique and, of course, has no other intersections with 
i 

Fix an affine chart such that I'(yo) = 0 and consider vectors v; = 
(P(y))™ lo=yo.t = 1,...,n. If they are linearly dependent, then the point 
I’(¥0) (or, for the sake of simplicity, the point (Yo) is called a flattening point 
of the curve IP’. Otherwise, the point Yo is called regular. 


Definition. An osculating hyperplane IT(¢o) at the point yo is any hy- 
perplane having intersection of order no less than n with the curve I at 


T'(¢o)- 
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Remark. 1. Osculating hyperplanes are well-defined. 

9. The osculating hyperplane I/(yo) at a regular point yo is the span of the 
vectors v1,... ,Un—1- The intersection’s order is equal to n in this case. 

3. The osculating hyperplane at an isolated flattening point is the limit of 
the osculating hyperplanes at nearby points 


Let. p(y) be the multiplicity of the intersection of I (yo) and I'(y) 
at the point yo. It is clear that p(y) = n at any regular point p. If 
u(y) =n+k , then k is said to be a flattening order of I at the point y. 
Evidently, a convex curve has no flattening points and it crosses IT(y) at 
I'(p) alone. 


Question (V.I. Arnold, [1]). Is the converse true? 


Remark. Another facet of a convex curve is the Chebysheff system of 
functions (cf. [1], [2]), considered up to transformations f;(p) > F'(¥) fil), 


where F'() > 0, and linear transformations f; > Gf, = 12 ee 
det(a?) 4 0. For example, the affinity of a convex curve (cf. below) is 
equivalent to the existence of a representative (1: fi :...: fn) in the 


corresponding class of Chebysheff systems. Only the segment J and the 
circle S! can be equipped with a Chebysheff system, cf. [2]. The case of 
a segment was explored by M.Z.Shapiro. See his Ph. D. thesis [3], where 
convex curves appeared as fundamental solution systems of disconjugate 
differential equations 


2. How Loss of Convexity Occurs 
The answer is furnished by the following 


Theorem 1. Let I;(y) be continuous (with respect to t) family of C?*1- 
smooth closed curves in RP™. Suppose all the curves I are convex when- 
ever t <0, but the curve I'(y) = Ip(y) is not conver. Then there exists a 
flattening point of order 2 or greater on I'(y). Moreover, every hyperplane 
that crosses (ip) more than n times (according to multiplicities) is an os- 
culating hyperplane at a certain flattening point of order no less than 2. 


Proof. The proof is by induction over the dimension of RP”. For n = 2 
this fact is well known, but let us start from n = 1. For n = 1 the statement 
is: “Suppose C?-smooth maps I(y) : S' — RP! are diffeomorphisms 
whenever t < 0 and I(y) is not a diffeomorphism. Then there exists a point 
Yo such that I’'(¢0) =I" (yo) = 0, but yi A v2 implies (v1) # F'(y2). 
Moreover, I"’(y) = 0 implies '"(y) = 0.” This fact is evident and its 
proof is omitted. 


We now need the following 
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Construction. Put A = I’(yo) or without loss of generality A = I°(0). 
All the straight lines incident to A form a projective space RP"™~1. So, 
the central projection 74: RP” — RP”~} with the center at A is defined. 
Denote 7(y) = ma(I'(y)) if gy 4 0 mod 27 and let (0) be the projection 
on RP”~} of the tangent line of '(y) at the point A. 


It is clear that 7(y) is a C"-smooth closed curve in RP"—!; if yp 4 0, 
the curve y(y) is C"*1-smooth. Furthermore, let 6 be a hyperplane in 
iP? sod Bb — (3) a hyperplane in RP” passing through A. By k(y) 
we denote the multiplicity of the intersection of @ and 7 at the point +(y) 
and by K (wy) we denote the multiplicity of the intersection of B and I at 
I'(y). Then k(yv) = K(y) for all p40 mod 2m and K(0) = k(0) +1. We 
obtain immediately 


Lemma. If the curve I'(y) is conver, then the curve y(¢) is also convez. 


Suppose the theorem is proved for RP, RP?, ... ,RP”~'; let us prove 
it for RP”. Assume the converse. Then the curve Io(y) is not convex and 
in RP” there is a hyperplane B that intersects ['(y) more than n times 
(according to multiplicities), but there are at least two different points of 
intersection. Recall also that all the curves I;(y), t < 0, are convex. It 
follows from here that there is a point A of multiple intersections; we can 
assume that A = I°(0). In fact, if all the intersections are transversal, then 
all the curves I,(y) with small |t| are not convex because they intersect B 
more than n times. 

Now consider the central projection 74: RP” — RP" }. We can 
suppose I;(0) = Ip(0) = A. Put %(y) = ta(i(y)). We know that all the 
curves 7(¢), t <0, are convex, but the curve Yo() is not convex and the 
hyperplane @ = 7,4(B) (it is really a hyperplane because A € B ) intersects 
y more than n— 1 times (according to multiplicities). Finally, there are at 
least two different points of intersection: (0) and the projections 74 of the 
other intersections of B and I’. This contradicts the induction hypothesis. 
The proof is finished. 


3. Affinity of convex curves in RP?” 


Definition. A subset M of a projective space P is called affine if there ex- 
ists a hyperplane JJ that has no intersections with M. So, if the hyperplane 
TZ is infinite, then the whole set M lies in the finite part of P. 


Theorem 2. Every convex curve I'(y) in an even-dimensional projective 
space RP?" is affine. 


Proof. First recall that T € C?"(S1,RP?"). Let A be an arbitrary point 
on I’, to be definite, A = (0). Put IJ = 11(0) (see §1). By the previous 
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construction, JT crosses I’ at the point A only and the order of this crossing 
equals 2n. Fix an affine chart such that the equation of [7 is ran =O and A 
is the origin. Consider the C?”-smooth function zan(I(y)). It has a zero 
of order 2n (but not 2n + 1) at the point y = 0. So, 


an 


<pitan(l(p))lo=0 #0 
and near the origin A the curve I lies at the one side of I7. We can suppose 
Zon(I'(y)) > 0 whenever y € (—6,6). Draw the hyperplane []-: t2n=—€ , 
where ¢>0. If is small enough, then I and /7, do not intersect: there are 
no intersections at the points I'(y), y € (—6, 6) by the construction and at 
other points of I” as well by upper semi-continuity of the crossing’s order. 
The theorem is proved. 


For a given convex curve Cc RP?" denote by M(I) c (RP?")* the 
set of hyperplanes having no intersections with I’. 


Theorem 3. The set M(I) is contractible. 


Proof. Fix an affine chart containing the whole curve I’. The existence of 
such a chart is provided by Theorem 2. Suppose the origin lies on I’. Then 
every hyperplane JT of the set M has the equation 


Sie where Sea and d>0 


(d40 because the origin does not belong to JZ since it lies on I’). Let I(¢) 
be the hyperplane defined by the equation 


= d (1+t0n +) : 


This formula prescribes a contraction of the set M to its single point (the 
infinite hyperplane). In fact, the substitution t = 1 yields the equation 
)~ ajz;= 00 for the hyperplane [7(1) and the described contraction is con- 
tinuous on M x [0,1]. So, the theorem is proved. 


4. Odd-dimensional case 


Let us consider a convex curve I’ in RP?"+! now. Choose 2n + 1 different 

points on I and draw a hyperplane JZ through chosen points. I" crosses 
IT 2n+1=1 mod2 times. So, [I] and [0] are two different elements of 
m™(RP?"*1). Since affine curves are homotopic to zero, any convex curve 
in RP2"+! cannot be affine. 


Theorem 4. There exist hyperplanes crossing the curve I’ only once. Such 
a hyperplane can pass through an arbitrary point of I. 
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Proof. The case n = 0 is trivial. Assume that n>0 and A is arbitrary point 
on I’. Put y= 7a(I) C RP?". The curve y is convex by Lemma from §2. 
It follows from Theorem 2 that there exists a hyperplane @ C RP?” having 
no intersections with y. Then the hyperplane B = 741(@) C RP?"*} 
crosses I’ transversally at point A and does not cross I’ at other points. 


Q.E.D. 


For a convex curve [ C RP2"+! we denote by M(I) c (RP2"*1)* 
the set of hyperplanes that cross the curve I only once (transversally, of 
course). 


Theorem 5. The set M(I) is contractible to S1; this circle is not homo- 
topic to 0 in P = (RP2"*1)*. 


Proof. The first idea is to note that the image of the natural map ®: M — 
S! defined by the rule ®(J7) = [-1(ITNI'(y)) is S! and the preimage of 
each point y is contractible by Theorem 3. But this observation leads us 
to weak homotopy equivalence only. 


To prove the proclaimed theorem, let us fix a certain Riemannian metric 


on P. Put M, = 6—1(y). Evidently M, C RP” CP. 


Lemma 1. My is conver in any affine chart that contains the whole set 


My. 


Lemma 2. There exists a continuous “section” s: Si — M such that 
Gos =ids. 


Both proofs are straightforward. Note also that Lemma | gives another 
way to prove Theorem 3. 

Let p(z) be the distance between x and s(®(z)) along the segment 
[x, s(@(x))] = I, and let x, € I, be the point of J, such that p(zt) = 
(1—-t)p(x). Put F(z,t) =2,. Then F(z, t) defines the required contraction. 
The second statement of the theorem follows from the fact that for any point 
A € I the intersection index of the circle s(@) and the dual hyperplane 
A* € P is equal to 1 mod 2. 


5. Generalization: convex surfaces 
Definition. A smooth k-dimensional surface T* embedded in RP” is called 
convex if every plane of codimension k intersects I’ no more thann—-—k+1 
times. 


Example. Every point is 0-dimensional convex surface. Every domain 
in RP” is n-dimensional convex surface. Convex curves (cf. above) are 1- 
dimensional convex surfaces. Spheres in R” c RP” are (n — 1)-dimensional 
convex surfaces. 
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For hypersurfaces our definition is the same as that of the classical one. 
So it can be proved that every convex hypersurface in RP” is affine and is 
homeomorphic to S"~1 (cf. [4], for example). 


Theorem 6. There are no convex surfaces I*® in RP” if k = 2,3,...,n—2. 


Definition. A smooth k-dimensional (k > 0) surface [* C RP” is called 
almost convex if there exists a unique plane JJ*-1 c I’* and every plane 
Bn- intersects I at most n—k-+1 times whenever dim(B"~*NH*~*) <1. 


Lemma. Almost convex surfaces of codimension 1 do not exist. 


Proof. The case k = 1 is evident: any point A € I is a plane of dimension 
k—1. So, suppose k > 1. Let A be an arbitrary point in Ee \ W* 1) Since 
k > 1, the tangent plane T4I° and I k-1 have non-empty intersection. If 
A € Tal MN T*!, then the straight line AA’ intersects I’ at least three 
times: twice at A and once more at A. So the surface I* is not an almost 
convex surface. 


Proof. (of the theorem) Consider the central projection 74: RP” — 
RP"-!, where A€ I. Put y= 7a(L). It is easy to prove that y C RP”“" 
is an almost convex surface of dimension k and codimension n — k — 1. It 
contains the plane 17*-! = m,4(A). Next, consider the central projection 
T,: RP"-! — RP*-?, where a € I*-1 Cc y. Then ma(y) C RP"? is 
an almost convex surface of dimension k and codimension n — k — 2, and 
so forth. Finally, we get an almost convex hypersurface in RP*+!. This 
contradicts the previous lemma. The theorem is proved. 


Remark. We have obtained in the proof of Theorem 6 that there are 
almost no convex surfaces of any codimension. 


-The author is grateful to V.I. Arnold, V.A. Vassiliev, A.G. Khovanskii 
and M.Z.Shapiro for useful discussions. 
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Topological classification of real trigonometric 


polynomials and cyclic serpents polyhedron 


V. Arnold 


The goal of this paper is the study of the manifold of the real trigono- 
metric polynomials of degree n having the maximal possible number (27n) 
of real critical points (M-polynomials). 

We shall find the number of the connected components of the space 
of the generic M-polynomials, having 2n different critical values. We con- 
struct a polyhedral model of the manifold of M-polynomials and a real 
algebraic diffeomorphism sending the manifold of M-polynomials onto the 
interior of a convex polyhedral cone over the product of two simplices of 
dimension n — 1 and of a line. Those polynomials, which are not generic, 
are sent onto some diagonal hyperplanes of this polyhedral cone. This dif- 
feomorphism can be continued as a homeomorphism up to the boundary 
of the cone (and defines the diffeomorphisms on the interior parts of the 
boundary faces of all dimensions). 


1. The space of cyclic serpents 


Definition. A cyclic serpent of order 2n is a sequence of 2n real numbers, 
verifying the inequalities 


Co < cy & C2 <1... 2 Con = CO. 


The serpent is proper, if all the inequalities are strict, and generic, if 
all the 2n numbers are different. 

The space of the cyclic serpents of order 2n is a polyhedral convex cone 
in R2". The hyperplane c) + ... + Con = 0 intersects it along the strictly 
convex cone of traceless cyclic serpents. The cone of all cyclic serpents is 
the product of the cone of the traceless cyclic serpents with R. 


Definition. The intersection of the cone of the traceless cyclic serpents of 
order 2n with the hyperplane R2”~?, where (—1)'t!c¢; = a> 0, is called 
the cyclic serpent polyhedron Py. 

It is a compact convex polyhedron in R?"~?. 


Example. P» is a square in the plane. The interior of a square is diffeo- 
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morphic to the domain, bounded by an astroid (which is the caustic of an 
ellipse). The diagonals of the square are sent by a natural diffeomorphism 
to the Maxwell stratum (consisting of two segments joining the opposite 
cusps of the astroid). 

The diffeomorphism of the serpent polyhedron onto the space of 
trigonometric polynomials that we shall construct below is a higher- 
dimensional generalization of this combinatorial description of the astroid 
caustic and of its Maxwell stratum. 


Theorem 1. The cyclic serpent polyhedron P,, is a convex polyhedron in 
R2"-2 | having 2n faces and n* vertices : the direct product of two simplices 
in R™}. 


Proof. Let cox41 — Cok = Qk , Cok4+1 — Con42 = by (k = Oj ccogtt = 1). The 
equation of the hyperplane R?"~? implies the conditions La, =a, Lb; = a. 
The cyclic serpent definition implies the inequalities a, > 0, b, > 0. Any 
solution of these equations and inequalities defines exactly one traceless 
cyclic serpent belonging to P,,. Hence P, is the product of the simplices 
Naz, =a, a, > Oand Db; = a, by > O in the hyperplanes of dimension n—1 
defined by the equations Na, = a , “by =a in the n-dimensional a-space 
and b-space. The 2n faces are defined by the equations a, = 0, b, = 0. 


Now we describe the diagonal hyperplanes of the product of two simplices, 
defined by the conditions c; = c; in the original coordinate system in the 
serpent space. 

Write the cyclical word A; By A B3...AnBn(AiB}...). Decompose it into 
two connected arcs. Each decomposition defines a diagonal hyperplane in 
the product of two simplices with vertices (A; A2...A,) and (B,B3...B,). 


Definition. The principal diagonal of P,, is the hyperplane containing the 
vertices (A;, B;) where A; and B; belong to the same arc. (Here A, is the 
vertice of the simplex in R”~! described above, defined by a, =a, By, - 
by by = a). 

The number of such diagonals is equal to n(2n—1) and they are defined 
by the equations c; = c; in the original coordinates. 


Theorem 2. The principal diagonal hyperplanes subdivide the polyhedron 
P,, into the simplicial domains. The number of these domains is equal to 
mi,, where the tangens ambvers) ly) — 12, llb 272. — enone nane 
defined by the exponential generating function 


[oe] 
SS oes /(en- i = inees, 
n=1 


Example. For n = 2 we get 27, = 4. Two diagonals cut a square into 4 
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triangles. 


Proof. It suffices to study the components of the space of all cyclic serpents 
(the trace zero and scaling restrictions being irrelevant). The principal di- 
agonal hyperplanes are defined by the equations c; = c;. Their complement 
is the space of the generic serpents. 


Construct a mapping from the space of generic cyclic traceless serpents 
of order 2n onto the space of the generic ordinary serpents of order 2n — 1. 
Delete the smallest of the numbers c; (say, cox). Rename the remaining 
numbers, denoting them by 


bs = C2k-+i(mod2n) » 1 eee — 1. 


The sequence b, > bz < bg > ... < ben—1 is an ordinary serpent on the ray 
(coz, +00), diffeomorphic to R. 

We have thus constructed a locally diffeomorphic mapping from the 
space of the (virtually nongeneric) traceless cyclic serpents of order 2n onto 
the space of the (virtually nongeneric) ordinary serpents of order 2n — 1 
in a neighbourhood of every point c, at which the minimum of the ¢,-s is 
attained only once. This diffeomorphism sends the generic cyclic serpents 
to the generic ordinary ones. It follows that the components of the manifold 
of the generic traceless cyclic serpents are simplicial cones (as well as the 
components of the space of the ordinary serpents). The components of the 
complement to the principal diagonal in P,, are hence open simplices. 

The number of such simplicial cones (simplices) is equal to the prod- 
uct of the number of the components of the space of the ordinary generic 
serpents of order 2n — 1 (which is equal to T,,, see for example [1]) and of 
the number n of the possible values of k =1,...,7, defining the minimum 
Ck = min Cj. 


2. The space of real trigonometric M-polynomials 


We normalize the trigonometric polynomials of degree n fixing the high- 
est term and removing the constant term, 


FG) OSE tp DE, (a, cos kt + by sin kt). (1) 
O0<k<n 


We consider the space of these polynomials as the real linear space R?"~? 
with coordinates (ax, x). 


Definition. An M-polynomial is a trigonometric polynomial of degree 
n having no complex critical points (the 2n real critical points might be 
degenerate). It is proper, if all the 2n critical points are different (and hence 
nondegenerate). It is generic, if all the 2n critical values are different. 
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The M-polynomials (1) form a real semialgebraic set in the space 
R2"-2_ The main result of the present paper is 


Theorem 3. The set of M-polynomials (1) in R2"~? is homeomorphic 
to the cyclic serpent polyhedron P,. There exists a semialgebraic home- 
omorphism, inducing the diffeomorphism of the space of the proper M- 
polynomials on the interior domain of P,, and sending the nongeneric M- 
polynomials onto the principal diagonal hyperplanes of the polyhedron P,. 


Corollary. The connected components of the space of the generic M- 
polynomials (1) are contractible and their number is equal to nT,,. 


Example. For n = 2 the manifold of the proper M-polynomials (1) is 
the domain of the (aj, b;)-plane, bounded by an astroid. The generic M- 
polynomials form four triangles, into which the domain is cut by the two 
segments, joining the opposite cusp points. This decomposition is diffeo- 
morphic to the decomposition of the open square P2 into four triangles by 
its diagonals. 


Proof. The complex version of the required mapping is constructed in [2] - 
it is the L3 (Lyashko-Looijenga-Laurent) mapping. The real version for the 
ordinary polynomials case is described in [3] as the real version of the L2 
(Lyashko-Looijenga) mapping. The polyhedral model, similar to our Pp, is 
in this case the Springer cone. 

To adapt these constructions to the trigonometric case one uses the 
following fact (extending the Vacoulenko construction described for the 
ordinary polynomials in [3]). 


Lemma. Every cyclic serpent of order 2n is, up to an additive constant and 
to the multiplication by a positive number, the sequence of the consecutive 
critical values of a real trigonometric M-polynomial (1). 

This polynomial is defined by the serpent uniquely, up to the shift of 
the argument t by a multiple of 27 /n. 

We shall prove this for the proper serpents, the proof in the nonproper 
case being similar. 


Proof. Take 2n copies X; of the (extended) complex plane of c (j = 
0,...,27 ; Xo, = Xp). Cut Xo, along the real axis from co, to —oo 
and from ¢2441 to +00, and X2x41 from cg,41 to +00 and from co442 to 
—oo. Glue the opposite sides of the common cut (c;41,(—1)J00) on X; and 
on AGG = (0). 1, 2 — De 

The Riemann surface X thus constructed is a sphere (according to the 
Riemann-Hurwitz formula). The point c = 00 is covered by two points of 
the surface X, each with multiplicity n. 

The Riemann theorem on the unicity of the complex structure on the 
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sphere implies that the projection of the surface X onto the c-plane can be 
written as a Laurent polynomial 


Cr f(z = Anz ee ne 


where z is a coordinate on X, such that the real axis on X (the fixed points 
set of the antihomomorphic involution, covering the complex conjugation 
Olc) Is) — 

The reality condition implies that f(1 /Z) = f(z). Thus f is a real 
trigonometric polynomial of ¢ where z = e’*. Choosing the origin of t we 
can reduce the highest degree term toa cos nt, a > 0. 

We have thus constructed the required polynomial. Its unicity follows 
from the fact, that the topological type of the covering is unambiguously 
defined by the serpent (see [2]). Hence all the real trigonometric polyno- 
mials defining the same serpent may be obtained one from the other by an 
automorphism of the sphere X. 

The automorphisms, sending 2” + z~” to itself, are z +> €z and z > 
€ Hl z, where e” = 1. The second type automorphism (say with e = 1) would 
transform the serpent co < ¢1 2 C2 S<.-.- int@cp Ss oni On-2) = =: 
We deduce that cy. = Can—z~ and hence f is even and invariant under the 
automorphism. Thus only the translations of t by 27k /n remain, and the 
Lemma follows. 

To prove the theorem, we kill the constant term in the polynomial 
(adding a constant to the critical values). Multiplying f by a positive 
number , we reduce the coefficient a of cosnt to one. We have thus con- 
structed a diffeomorphism of every connected component of the manifold 
of the generic M-polynomials (1) onto one of the simplices into which the 
polyhedron P,, is cut by the principal diagonals. 

This simplex is defined by the generic cyclical serpent of the critical val- 
ues only up to the action of the cyclic group of order n, since the polynomial 
does not know which of its local minima is Co. 

The component of the space of the generic M-polynomials (1) is defined 
by the cyclic serpent also only up to the action of the cyclic group (of the 
shifts of the argument t by the multiples of 27 [?). 

Both ambiguities are, however, cocordant. 

Choose any pair (component, simplex). This choice defines an Z,- 
equivariant diffeomorphism of the manifold of the proper M-polynomials 
(1) onto the interior domain of the cyclic serpent polyhedron P, (this dif- 
feomorphism is the real version of L3). 

Indeed, consider a generic path, connecting a generic M-polynomial (hy) 
with its version, shifted by 27k ij n in the space of proper M-polynomials. 
This path is covered by a well-defined generic path in the proper serpent 
space, connecting the serpent of the critical values of the original M-polyno- 
mial to the shifted serpent. 
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A generic path, connecting a generic serpent with its shifted version 
in the space of proper serpents, defines unambiguously a generic path in 
the space of the proper M-polynomials (1), connecting the original generic 
M-polynomial (1) with its shifted version. 

Hence we obtain a homeomorphism of the space of the proper M- 
polynomials (1) to the interior domain of the principal cyclic polyhedron 
P,.. This homeomorphism is a diffeomorphism, since, according to [2], 
the critical values are the local coordinates in the space of the proper M- 
polynomials (1). 
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Singularities of short linear waves 
on the plane 


Ilia A. Bogaevski 


ABSTRACT. The subject of the paper is the geometrical optics of short lin- 
ear waves on plane. We describe perestroikas of momentary fronts and 
scattering of rays when the light hypersurface has generic conical singulari- 
ties. They appear if the waves propagate in a nonhomogeneous anisotropic 
medium and are controlled by a hyperbolic variational principle. 


The geometrical optics of short linear waves on plane is described by 
some hypersurface in the contact space of the projectivised cotangent bun- 
dle of space-time. This hypersurface is called light. Projections of Le- 
gendre submanifolds of the light hypersurface define in space-time surfaces 
called big fronts and describing the wave propagation — their sections with 
isochrones are momentary fronts evolving in time. Big fronts consist of rays 
which are projections of characteristics of the light hypersurface. Usually 
the light hypersurface is evolutionary. It means the rays are transversal 
to the isochrones. In this case the evolution of a given momentary front 
is completely defined by some initial condition which is an integral (with 
respect to the contact structure) curve on the light hypersurface. Namely, 
the corresponding Legendre submanifold is the extension of the initial con- 
dition along characteristics of the light hypersurface and the projection of 
the initial condition is the given front. 

A momentary front evolving in time can experience perestroikas (= bifur- 
cations, metamorphoses). In case of a smooth evolutionary light hypersur- 
face and a typical initial condition all perestroikas of momentary fronts are 
well known even if their dimensions does not exceed four (see, for example, 
(1), (21, (3). 

However, if the oscillations in the waves have more than one degrees of 
freedom, on the light hypersurface there can be conical singularities ([4]). 
Let the initial condition be smooth and do not pass through any singularity 
of the light hypersurface. Then it is called regular. But since some char- 
acteristics can pass through singularities of the light hypersurface, even in 
that case the extended Legendre submanifold can became singular too. Its 
singularities generate new singularities of big fronts and new perestroikas 
of momentary fronts. They are found in book [4] in case of a typical evo- 
lutionary light hypersurface having only conical singularities and a typical 
regular initial condition. However, the proposed there description of the 
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corresponding singularities of big fronts and perestroikas of momentary 
fronts is not correct. 

Our results imply there are two types of the new perestroikas of momen- 
tary fronts. These types we call refraction and reflection of waves and show 
in Fig. 1. 


Figure 1. Refraction and reflection of waves. 
Perestroikas of fronts and scattering of rays. 


In the both cases one ray consists of two branches of intersecting smooth 
curves. These branches are the incoming ray and the outgoing one. The 
other rays are smooth and scatter. Namely, if € is the distance between a 
fixed point of the incoming ray and a smooth ray, the distance between the 
same ray and a fixed point of the outgoing ray is equal to we Ine+o(e Ine) as 
€ — Oand uw <0. At the perestroika instant the momentary front consists 
of two tangent branches having different curvatures at the point of their 
tangency. Each of them can be given in suitable smooth coordinates by the 
local equation y = x*/Inz + o(x?/Inz) as x — 0. After the perestroika 
instant the momentary front acquires a discontinuity of the third derivative. 
The second derivative proves to be continuous but not to satisfy the Hélder 
condition. The discontinuity propagates along the outgoing ray and in 
its neighborhood the momentary front can be given in suitable smooth 
coordinates by the equation y = z*/In|z| + o(x?/In|z|) as x > 0. 

In the case of reflection the cusp of the momentary front propagates 
along the caustic. It is shown in Fig.1 by the double line. When the 
momentary front experiences a perestroika, the caustic has a discontinuity 
of the second derivative. At this point the caustic’s curvature is infinite 
and changes a sign. The caustic can be given by the local equation y = 
(A — sign x)z/ In |r| + o(2/In|z|) as x > 0 and A> 1. 


1. Singularities of Light Hypersurfaces 
A singular point of a light hypersurface 


GC Pig 
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in the contact space of the projectivised cotangent bundle of space-time 
R2+! is called conical if in its neighborhood %* has the local equation 
u2 + v2 = w? in suitable smooth coordinates. There are two types of 
generic conical points of light hypersurfaces with respect to the contact 
structure in PT*R2+! (see [4], [5]). Namely, a conical singularity of the 
light hypersurface ¥4 is hyperbolic if it has the local equation P,Q, 4+ P2 = 
0 in suitable formal coordinates (P;, P2,Qi1,Q2,U) such that the contact 
structure is given by the equation 


= 5(PaQ _~ QdP). 


If 54 is given by the local equation 
Pi + Qi =P; 


then its conical singularity is called elliptic. 

A characteristic of ©4 does not pass through any its elliptic singularity. 
On the contrary, there exist a couple of characteristics passing through each 
hyperbolic singularity of y4. Their equations in the above coordinates are 
P, = Pp =Qo =U =0 and P2 = Qi = Q2 =U = 0. 

It turns out there are three types of generic hyperbolic singularities of 
light hypersurfaces with respect to the natural projection 


Piso 


Namely, let us say these types are refraction, reflection, and pseudoreflection 
of characteristics. They can be defined if the two following conditions of 
regularity hold at the considered hyperbolic singularity. 

1) The projection of the surface of all hyperbolic singularities into space- 
time has a full rank. 

2) The projections of the directions of the two characteristics passing 
through our hyperbolic singularity are different. 

Let H2 c XS be the surface of all hyperbolic singularities of the light 
hypersurface, N4 = n—1((H?)), and H? c &* be the union of all charac- 
teristics passing through H?. Locally H? = H}UH3 and H* =H 383, 


where the local components H ® and H @ are given by the equations 
P,=P,=0 (H}) and P=, =0 Ge), 


All submanifolds 54, N‘, H 3, and H @ contain H 2. The refraction, reflec- 
tion and pseudoreflection of characteristics are all different variants of the 
arrangement of these submanifolds providing N 4 is tangent to none of the 
submanifolds D%, H p, and H a at the considered point. These variants are 
shown in Fig. 2. 
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Figure 2. Points of refraction, reflection, and pseudoreflection of characteristics. 


In space-time there exist smooth coordinates (y, z) such that the projec- 
tion a: (P,Q,U) & (y, z) is given by the asymptotic formulas 


yi =aP,+bP,—-Qit..., 
yo = bP cl, — Qa. , 
gE eProp WE 


where the dots denote the terms of higher degrees providing deg P = 
deg @ = 1 and degU = 2. Then 

a <0 and b ¥ 0 in the case of refraction of characteristics at the origin 
of the coordinates (P,Q,U); 

0 < 4a < b? in the case of reflection of characteristics; 

0 < b? < 4a in the case of pseudoreflection of characteristics. 


2. Singularities of Big Fronts 


Let L1 c %* be a regular initial condition. Its nontrivial intersection at 
isolated points with the submanifold H? c ¥4 (which consists of all char- 
acteristics passing through the surface H? Cc ¥*4 of the hyperbolic singular- 
ities) is a typical phenomenon. Let our initial condition be transversal to 
H?. Then the extended Legendre submanifold L? can intersect the surface 
H? at isolated points. The singularities of L? at these points are described 
by the parametric normal form 


P,=A, P,»=AB, Q,=-B?, Q2=-2A4BInA?, U =—A?B?/2 


found by V.I. Arnold in [4] (A, B are the parameters). The normal forms of 
the contact structure and the light hypersurface remain the same as before. 
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By substituting these equations into the asymptotic formulas for the 
projection a from the previous section we obtain the local asymptotic 
parametrization of the big front ( L7) 


y, = aA?+bAB+B?+..., 
y2 = bA? +cAB+2ABln A? +..., 
z= aA*/2+bA°B + (1+ c/2)A°B? + A°B? In A*+.... 


By the change of parameters A ++ aA, B + GB one can achieve a = +1, 
b>0,c=0. Then a = —1 in the case of refraction of characteristics, a = 1 
and b > 2 in the case of reflection of characteristics, a = 1 and b < 2 in the 
case of pseudoreflection of characteristics. The singularities of big fronts 
and the rays on their in these three cases are shown in Fig. 3. 


Figure 3. Refraction, reflection, and pseudoreflection of rays. 
Singularities of big fronts 


3. Perestroikas of Momentary Fronts 


If our light hypersurface is evolutionary then the pseudoreflection of 
characteristics is not realized. The rest singularities of big fronts describe 
the refraction and reflection of momentary fronts. Namely, the momentary 
fronts are the sections of the big front with the isochrones and transversal 


to the rays. 
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New generalizations of 
Poincaré’s geometric theorem 


Yu. V. Chekanov! 


Poincaré’s geometric theorem claims that an area-preserving diffeomor- 
phism of an annulus which shifts the boundary circles at opposite directions 
has at least two fixed points. The present paper consists of two parts. In 
the first one, we show that such a diffeomorphism has more than just two 
fixed points provided the shift of the boundaries is large enough. In the 
second part, we prove symplectic fixed point theorems which can be viewed 
as generalizations of Poincaré’s geometric theorem to higher dimensions. 


ils 


Consider the annulus A = [a;,a_] x S', where S 1 — R/Z, endowed with 
an area form w. Let g be an area-preserving diffeomorphism of A. 

Denote by g the diffeomorphism of A = [a_,a4] x R which covers g. 
Define the shift functions p+:R — R by 


g(a, y) = (ax, y+ y+(y))- 


Poincaré’s geometric theorem (actually proved by Birkhoff [1]) claims 
that if p, > 0 and y_ <0 then g has at least two fixed points. 
We prove the following stronger result. 


Theorem 1.1. If y4 > Ny and y_ < N_, where Nz€Z,then g has at 
least 2(1 + Ny — N_) distinct fixed points. 


In the middle of 1960s V. I. Arnold demonstrated that Poincaré’s ge- 
ometric theorem is a consequence of the fact that any area-preserving dif- 
feomorphism of a 2-torus which is a time 1 map of a (non-autonomous) 
Hamiltonian flow has at least three fixed points. He formulated a conjec- 
ture estimating from below the number of fixed points of a Hamiltonian 
symplectomorphism of an arbitrary symplectic manifold [1,2]. Later C. 
Conley and E. Zehnder proved the Arnold conjecture for standard sym- 
plectic tori [4]. 

We show Theorem 1.1 to follow in a quite elementary way from the 
Conley-Zehnder theorem for the 2-torus. The proof of Theorem 1.1 is 
based on the following 


0 EEE 
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Lemma 1.2. If py, >0 and y_ <0 then the projection 7: A-— A sends 
the set of fixed points of g to a set containing at least two points. 


Proof 1°. By the Moser theorem, we can assume the area form w to 
coincide with the standard area form dz A dy on the annulus 


A= (@,y) |e (a= raal ea) 2) 


Choose a smooth family {g*}, t € [0,1] of w-preserving diffeomorphisms 
of A in such a way that, for some covering family {g} of diffeomorphisms 
of A, which preserve the form @ = 7*(w), the following holds: 

g@ =id, 9 = 9, 9 (Gi,4) — (ae tet Gy): 

(To do this we first construct such an isotopy in the class of all diffeo- 

morphisms, then apply the 1-parametric version of the Moser theorem.) 


2°. Let {h*}, t€ [0,1] be the family of Hamiltonians on A generating 
the flow {g*}. It means that dh’ = i,,w, where v4; = 09'/Ot. The form dh 
is clearly invariant under the action of the covering transformation group 
Z on A. Since the vector field 1; is tangent to aA, the restriction of h to 
either component of 0A is a constant function for any té [0,1]. Therefore 
the Hamiltonians h* are Z-invariant. 


3°. Since the functions h* are defined up to additive constants, we can 
assume them to be positive. Let 64, ¢ > 0, be smooth monotone functions 
on [a_, a4] such that 64 = 0 on [a_,a,—2e], 64 = 10n [a,—¢,a,], B& =0 
on [a_+2¢€,a4], 8° =1 on [a_,a_+e]. Define a family of functions on A 
by 


he(a,y) = B4.(z)(a4.— 2) + BE (a)(a — a_) + (1— 64 (a) — B2(2))h*(2,y). 


Since the Hamiltonians ht are Z-invariant, we have ht = ht o7 for some 
family {h£}, t€ [0,1], of functions on A. Consider the flow {gt} on A as- 
sociated with the family of Hamiltonians {h¢}, and the flow {g¢} on A 
associated with the family of Hamiltonians {ht}. 

For € small enough, the fixed points of g} are exactly the fixed points of 
g. To complete the proof of Lemma 1.2, it suffices to show that there exist 
at least two points z € A such that gi(z) = z and the loop {g¢(z)}, te [0, 1, 
is contractible because these points are exactly the images of fixed points 
of g} under the projection 7. 


4°. Consider the torus 


T = {(2,y) | c€R/(2a,—2a_)Z, yER/Z}, 
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equipped with the area form dx A dy as a double of the annulus A. More 
precisely, define [,,2: A — T by 


T(z,y) = ({z], y); Ih(z,y) a ((2a+ a a]; y); 


where [r] denotes the class of r in R/(2a,;—2a_)Z, then T = ,(A)UI2(A). 
Define the flow {Gt}, t € [0,1], on T by doubling the flow {gz} on A. 
More formally, 


Gt(z) = I; 0 gf 0 Ij *(z) when ze I;(A). 


This flow is defined by the family of Hamiltonians {H!}, where H(z) = 
ht o I7*(z) when z€ 1;(A) and Ht(z) = —ht o Iz *(z) when z€ Io(A). 

Let N denote the number of points z €T such that G2(z) = z and the 
loop {Gt(z)}, t€[0, 1], is contractible. By the Conley—Zehnder theorem, 
N > 3 [4]. Hence there exist N/2 > 2 points z€ A such that g}(z) = z and 
the loop {g¢(z)}, t € [0, 1], is contractible. This completes the proof. a 


Proof of Theorem 1.1 Using the action of the covering transformation 
group Z on A, one easily constructs 1+ Ny — N_ different diffeomor- 
phisms of A which cover the diffeomorphism g and satisfy the hypothesis of 
Lemma 1.2. Their fixed points project to A disjointly. Hence the theorem 


follows from the lemma. F 


2 


A. Floer introduced in [5,6] an elliptic version of infinite-dimensional Morse 
theory and used it to prove the Arnold conjecture for monotone symplectic 
manifolds. We use the same technique to obtain fixed points theorems for 
certain symplectic manifolds with boundaries. These results can be viewed 
as generalizations of Poincaré’s geometric theorem and Theorem 1.1. 

Let g be a Riemannian metric on a manifold M. Define the function 
L,:T*M — R to measure the lengths of covectors in the metric g* dual 


to g: 


L(p) = |p| = V9"(p,p) for pEeT MM. 


Denote Aj(M) ={reET*M | Loe) et. 

Fix a flat metric g on T” = R"/Z” and let H: (0, 1) x A,(T”) — R be 
a time depending Hamiltonian on A,(T™) coinciding with L, in a neigh- 
bourhood of 0A,(T”). Define G to be the time 1 map of the Hamiltonian 
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flow associated with H. Let N(g) denote the number of elements of 7(T”) 
represented by closed geodesics of length < 1 in the metric g (constant 
loops are assumed to be geodesics throughout). 


Theorem 2.1. Symplectomorphism G has at least 2”.N(g) fixed points 
provided all of them are non-degenerate. 


Consider the case n = 1 to describe the relation between this result 
and Theorem 1.1. For n = 1, A,(T”) is just the annulus. The metric g 
has form c(dq) @ (dq), where q is the natural coordinate on T! = R/Z. 
Assume 0 < No < 1/c < No +1 for some No € Z, then N(g) = 2No +1. 
One easily checks that G satisfies the hypothesis of Theorem 1.1 with Ny = 
(N(g) —1)/2, N- = (1 — N(g))/2. Hence the case n = 1 of Theorem 2.1 is 
a corollary of Theorem 1.1. 


Proof 1°. The function L, on T*T™” defines outside the zero section the 
Hamiltonian system describing the velocity 1 motion on T”: 


p=0, g=%p)/|p|, (2.1) 


where 7 : (R”)* — R” is the isomorphism defined by g). The projection 
T*T” — T” sends time t¢ trajectories of this system to length t geodesics. 
If there exists a length 1 closed geodesics on J”, then there are infinitely 
many trajectories of (2.1) projecting to this geodesics which lie in OA,. 
Then G has infinitely many fixed points and the claim of Theorem 2.1 is 
trivially satisfied. So we assume that there are no such closed geodesics. 

Extend H to a function on (0, 1] x T*T” by letting H(t, -) = L, outside 
A,(T;,). Assume now that H is actually defined on (S' = R/Z) x T*T” 
instead of [0,1] x T*T” (one can check that the general case follows from 
this one). The 1-periodic solutions of 


“(t) = sgrad H(t, (2(t)) (222) 


(sgrad denotes Hamiltonian vector field) are in one-to-one correspondence 
(given by x +> «(0)) with the fixed points of G. Our goal is to estimate from 
below the number of such solutions. The desired estimate will be obtained 
by means of the appropriate version of Floer homology. 


2°. Fix a connected component of the loop space C%(S!, 7”) contain- 
ing a closed geodesics of length < 1 (there exist N(g) such components). 
Denote by 92 the corresponding connected component of C*(S DEEN 

Define S(Q, H) to be the set of 1-periodic solutions of (2.2) belonging 
to 2. To prove the theorem, it suffices to show that $(Q, H ) contains at 
least 2” loops if they all are non-degenerate. 
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Consider the symplectic action functional Ay:C™(S!,T*T”) — R, 


a 
Au(«) = i Ee [ H(t, (2(t)) dt. 


The critical set of its restriction to N is S(Q,H). Endow T*T” with the 
complex structure J, 


J(8/0q:) = 0/0p;, J(0/Op;) = —0/0q;. 


The symplectic form w = dp A dq and the complex structure J define the 
metric w(J-,-) on T*T” and the metric 


1 
[ 2@,- ae 
0 
gmc °(S", 7 awe 
The (anti) gradient flow equation for the functional Ay reads 


du(s) 
ds 


(t) = ae" ~ VH"(u(s)(t)). (2.3) 


For z_,24 €S(Q, H), define M(xr_,x+) to be the set of gradient tra- 
jectories (solutions of (2.3)) u: R — satisfying the asymptotic conditions 
lims—+o0 u(s) = T+. 

One can prove that the collection of the points u(s)(t) over all 
we M(z_, 2+), 4 €5(0, A), ER, te S! form a compact subset of T*7” 
(for a similar case, the proof has been carried out in [8]). Therefore, in 
view of the fact that w is exact and hence no bubbling can occur, we can 
immediately apply the results established by Floer for the case of closed 
symplectic manifolds. This yields the following. (1) To any non-degenerate 
z € S(Q, H), one can attach the integer ind (x) called the (relative) Maslov— 
Conley—Zehnder index. 

These indices are defined up to a uniform additive constant. As- 
suming all critical loops to be non-degenerate, we get a grading (de- 
fined up to a shift) on the chain space CF,(Q,H) spanned over Z2 by 
the points of $(Q,H). (2) After a generic perturbation of H, any set 
M(az_,2+) is a manifold of dimension ind(x_) — ind(x). (3) The map- 
ping 0: CF,(9, H) > CF, _1(Q, H) given by 


az)= So #(M(z,y))y 
y €S(9,F), 
ind(y)=ind(z)—1 
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where c€S(0,H), M(z,y) = M(z,y)/R (R acts by translations in s 

variable), is a differential. The resulting homology HF,(Q,H) does not 

depend on the choice of (generic) H as long as H(t, -) coincides with L, 
outside a compact (or differs by a C?-small function). 


3°. To prove Theorem 2.1, it suffices to show that dim HF,(Q, H) = 2” 
(since this implies dimCF,(Q,H) > 2”). We construct a special H to 
symplify the computation of the homology. Let A be the length of the 
longest geodesics of length < 1. Fix a smooth function o on {0,+o00[ such 
that o(r) = r?/2 for r <1, o(r) =r for r large enough, and o’(r) > X for 
r > 1. The Hamiltonian system associated with the function H(t, p,q) = 
a(|p|) reads 


p=0, q=o'(|pl)i(p)/Ipl- 


Its time ¢ trajectory starting at the point (p,q) projects to a length o’(|p|)t 
geodesics starting at the point g. The definition of o implies the critical set 
S(Q, H) to be an n-dimensional torus consisting of trajectories 


t+ (pe, qt+til(p)), 


where po € (R”)* depends on 2, and gq is an arbitrary point of T”. One 
can check that, for any r€S(Q,H), kerdAg(x) = T,S(Q, H) and hence 
the restriction of Aq to 2 is a Morse-Bott function whose critical set 
S(Q, H) is diffeomorphic to T”. Having in mind the analogy with the 
finite-dimensional (or Palais-Smale) Morse-Bott theory, we conclude that 
HF,(Q,H) = H,(T",Z2) (after a suitable choice of grading). Unfortu- 
nately, the Morse-Bott version of Floer homology has not been explicitly 
developed yet. So, to be more formal, we have to perturb H to make Ay 
a genuine Morse function. 


4°. First, we prove that the homologies HF,(Q,H) are isomorphic 
for all 2. Then we carry out the computation of HF,(,H) for the case 
where {2 = Qo is the component comprising constant loops, H is some 
time-independent function. 

Consider the mapping 


AG Se f= Gea te (t, p,q) > (t,p + pg, q+ ti(pg)). 
For any t€ S', Xo(t,-) is a symplectomorphism. The trajectories of the 


Hamiltonian system associated with H are sent by Xp to the trajectories 
of the Hamiltonian system associated with H® defined by 


Hp q) = H(t, p— pg, q—ti(pg)) + ba(p), (2.4) 
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where bo(p) = g*(p, Pg) = (p,i(pg)). Since Xe(t, -) preserves J, the map- 
ping XQ yields a one-to-one correspondence between the gradient trajecto- 
ries (2.3) for Ay and that for Aye. Let H.(t,p,q) = o(|pl) + ev(t, p,q), 
where ¢ #0, y is a generic C?-bounded smooth function. 

Consider the gradient trajectories for the restriction of Ay, to 2. The 
weak compactness argument [6,7] shows that, for ¢ small enough, their 
images are contained in a neighbourhood of Tg = {p = pg}. 

Define 


H(t, p,q) = o(|pl) + e9(t, p — Po, q + ti(pq)). 


Since in a neighbourhood of Tp the Hamiltonians H® (obtained by applying 
(2.4) to H,) and H, differs by a constant, Xp yields (for € small enough) 
the one-to-one correspondence between the gradient trajectories for the 
restriction of Ay, to Qo and the gradient trajectories for the restriction of 
Aj, to 2. Hence (for ¢ small enough) Xq induces an isomorphism between 
TEL (Qo. wand A (0, H2). 

Choose now H, to be time-independent: 


H(t, p,q) = Fe(p, a) = o(lpl) + ef (4), 


where f is a generic Morse function on T”. One easily checks that, for € 
small enough, $(Qo, H-) consists of constant loops: 


S(Qo, He) = {x | x(t) = (0,4), 4f(a) = OF 


As in [6,7,9], one shows that there is a natural one-to-one correspondence 
between the gradient trajectories (in the loop space of I°T’ ™) connect- 
ing the critical points of the restriction of Ay, to Qo and the gradi- 
ent trajectories (on T*T™) connecting the critical points of Fe. Though 
T*T™ is not a compact manifold (as the manifolds considered in [6]), Fe 
is a proper Morse function on T*7” and hence we get an isomorphism 
HF,(0, He) = (T*T", Zz). We get dim HF,(Qo, H-) = 2”, which com- 
pletes the proof of Theorem 2.1. a 


Consider now the case of an arbitrary manifold M and an arbitrary 
metric g on M. Provided there is no length 1 closed geodesics, one easily 
constructs the homology H F,(, H) for any connected component 22 of the 
loop space of T*M. The number of the periodic trajectories of the Hamil- 
tonian system coinciding with (2.1) outside a compact can be estimated 
from below by the dimension of H F,(Q, #) (in the non-degenerate case). 
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The problem is to compute the homology. In some particular cases, 
this can be easily done. For example, the claim of Theorem 2.1 remains 
true while g is deformed in such a way that no length 1 closed geodesics 
appear. Another result of this kind is the following. 


Theorem 2.2. Let g be a metric on a closed manifold M. Assume that 
each non-constant contractible geodesics has length > 1, and there exist N 
elements of ™(M) represented by exactly one closed geodesics of length < 1. 
If {H*}, te [0,1], ts a family of Hamiltonians on A,(T”) which coincide 
with Lg in a neighbourhood of OA,(M) then the corresponding time 1 flow 
map has at least dim H,(M,Z2)+2N fized points provided they are non- 
degenerate. 


Note that the hypothesis of this theorem holds for a negative sectional 
curvature metric as well as for a sufficiently small metric. 


Proof of this theorem essentially follows the lines of the proof of Theo- 
rem 2.1. 

For H = o(|p|), the functional Ay on 2 is a Morse—Bott function whose 
critical set is diffeomorphic to M when 2 consists of contractible loops, and 
to S' when 2 contains a non-constant geodesics of length < 1. This yields 
the desired estimate. x 


In the general case, it seems natural to conjecture that 
@,HF.(O, H) = H.(M, Zz) @ H.(S", Z2) @ Hs (Fi, Eo, Z2), 


where F, is the space of loops on M having length < 1, and Ep is the 
subspace of constant loops. 
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Explicit formulas 
for Arnold’s generic curve invariants 


S. Chmutov and S. Duzhin! 


Abstract 


We review the explicit formulas for Arnold’s generic curve invari- 
ants due to Viro, Shumakovich and Polyak and add some remarks 
concerning the invariants of spherical curves and curves immersed in 
arbitrary orientable surfaces. 


1. Statement of the problem 


A generic curve is a smooth immersion of the circle into the plane whose 
only singularities are transversal double points. Up to a diffeomorphism of 
the plane, how many generic closed curves are there? 

One can immediately invent a host of invariants for generic plane 
curves, for example, the total number of double points, the Whitney in- 
dex or the (unordered) sequence of edge numbers of the regions into which 
the curve divides the plane. 


Figure 1: Two different curves with an equal number of double points 
(= 4), equal Whitney numbers (= —3) and equal sequences of edge numbers 
GS sill We 


The problem we will discuss is the study of invariants and the classi- 
fication of plane curves and also curves immersed in orientable surfaces. 
Although this problem has a venerable history (it was first studied by 
Gauss), the most basic invariants of plane curves, Jt, J~ and St, were 
only discovered in 1993 by V. I. Arnold who used V. Vassiliev’s approach 
to topological invariants through discriminants and singularity theory. 


1Both authors acknowledge financial support from the International Science Founda- 
tion and the Russian Foundation for Fundamental Research. 
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The discriminant is the set of all nongeneric curves viewed as a singular 
hypersurface in the space of all immersed curves. It divides the whole space 
into a number of connected components. Two generic curves are equivalent 
if and only if they belong to the same connected component, i.e., there exists 
a path in the space of generic curves connecting one curve with another. 

From this viewpoint, the role of basic invariants should be played by 
functions which are constant on each connected component and have a pre- 
scribed behavior when the path intersects the discriminant. In the general 
position such a path meets the discriminant only at a finite number of its 
generic points. A generic point of the discriminant corresponds to a curve 
with only one generic singular point belonging to one of the three categories: 
a direct self-tangency, an inverse self-tangency or a triple point. 

Arnold’s seminal work [A1] gives an axiomatic description of the three 
invariants J+, J~, St which correspond to these types of singularities and 
prove their existence from the general properties of Serre fibrations. In 
1994, two sets of explicit combinatorial formulas for the Arnold invariants 
were found by O. Viro [V], A. Shumakovich [Sh] and M. Polyak [Po]. 

In the present text, we will give an overview of the work by Viro, 
Shumakovich and Polyak and add several observations on our own part that 
might shed more light on the meaning of these formulas and emphasize their 
relation with other viewpoints on the invariant theory of generic curves. 

Since we are interested in plane curves as well as in the curves immersed 
in surfaces, we will treat only closed curves — observing that, in the plane, 
a parallel and in many respects simpler theory exists for long curves ([Tab]). 

Now we state the problem in a more precise manner. 

Let 7 : S' + R? be a smooth immersion of the circle into the plane 
whose image is a smooth submanifold of R? everywhere except for a finite 
number of double points with a transversal intersection. The problem is to 
study the invariants and find a classification of such immersions (“generic 
curves”) with respect to an arbitrary diffeomorphism of the circle S! and 
an arbitrary diffeomorphism of the plane R?. 


OO 000 OD 
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Figure 2: Plane curves with 1 and 2 double points. 


Example 1. There are 2 nonequivalent plane curves with 1 double point 
and 5 nonequivalent plane curves with 2 double points (Fig. 2). 
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There are three nonequivalent variations of this problem: one can 
consider only orientation-preserving diffeomorphisms of the circle, or only 
orientation-preserving diffeomorphisms of the plane, or both (see [A1]). In 
this paper, we will only consider the totally nonoriented case. 

One can also study a similar problem for immersions of the circle in 
arbitrary 2-surfaces. For example, there exists a one spherical curve with 1 
double point, two spherical curves with 2 double points and one curve with 
2 double points on the torus which is not spherical (see Fig. 3). 


CO 000 «Ka GS 


Figure 3: Spherical and toric curves with 1 and 2 double points. 


Note that although in the case of spherical curves a complete invariant 
(Gauss diagram) is well-known, the study of the invariants which have 
a special behavior with respect to the discriminant is interesting in it- 
self. Of course, Arnold’s basic invariants for spherical curves can be ex- 
pressed through the Gauss diagram, but these expressions are quite non- 
trivial (Polyak’s formulas modified for spherical case, see last section). 


2. Gauss diagrams 


In this section, we will describe the classical construction of a Gauss 
diagram which gives an important combinatorial invariant of plane curves 
as well as of curves immersed in closed orientable surfaces. 

Let 7 : S! — R? be a generic curve with n double points. Denote the 
double points by n letters a1, @2,.--,4n and mark the preimages of these 
points on the circle by the corresponding letters with exponents +1 or —1 
according to the following rule. 

The natural counterclockwise orientation of the circle S! induces a 

direction for travelling along the curve. A passage of the double point is 
said to be positive if the traveller sees another branch of the curve as going 
from left to right. 
Definition 1 A Gauss diagram of order n 1s a circle with n distinguished 
ordered pairs of distinct points, considered up to an arbitrary diffeomor- 
phism of the circle and up to the operation of a simultaneous reversing of 
order in each pair. 

Gauss diagrams are in one-to-one correspondence with signed Gauss 
words, i.e., elements of the quotient set 


W(n)/Sn X Dan x Za, 
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where W(n) is the set of all permutations of 2n letters a1,a]',...,@n,a7! 
in which the symmetric group S$, acts by permutations of subscripts, the 
dihedral group Do, acts by cyclic shifts and reflections, and the nontrivial 
element of the group Zz changes the exponents of all letters. 

A Gauss diagram can be depicted as a chord diagram with oriented 
chords. The arrow on a chord goes from the positive to the negative passage 
of the double point. The simultaneous reversing of the orientations of all 
chords corresponds to an orientation-reversing diffeomorphism of the plane. 

The curves of Figure 1 have different Gauss diagrams: 


Figure 4: Gauss diagrams of the curves of Figure 1. 


The construction of the Gauss diagram G(T) is valid not only for a 
plane curve I’, but also for a generic immersion of the circle into an arbitrary 
orientable surface. 

There are three different Gauss diagrams of order 2 (see Fig. 5). Among 
them, two first diagrams correspond to plane curves, while the last one can 
only be realized by a curve in the torus (and hence in an arbitrary surface 
of genus higher than one). 


oO O © 
CD 68 


Figure 5: Curves corresponding to Gauss diagrams of order 2. 


In general, any arbitrary Gauss diagram corresponds to a curve im- 
mersed in a surface of a sufficiently high genus. This was proved by J. Scott 
Carter [C] who closely followed L. Heffter’s [H] classical construction of a 
closed oriented surface containing a given graph with a prescribed cyclical 
order of edges at every vertex. 


Theorem 1 (J. Scott Carter [C]) For any Gauss diagram G there is a 
minimal number g (the genus of G) such that G is the Gauss diagram of a 
certain generic curve in the closed orientable surface of genus g. This curve 
is defined uniquely up to a diffeomorphism of the surface. In other words, 
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the Gauss diagram constitutes a complete invariant for those curves tm- 
mersed in a surface that cannot be immersed in a surface of smaller genus. 
In particular, the Gauss diagram of a plane curve is the complete invariant 
of its spherical class. Moreover, if two immersions of S! in two different 
surfaces Q1, Qo have the same Gauss diagram, then one can remove some 
handles from either or both of the surfaces Q1, Qo so that the resulting 
immersions will be equivalent. 


The minimal surface of a Gauss diagram can be obtained by attaching 
disks to the curve viewed as a 1-skeleton in such a way that in the double 
points these pieces are patched together consistently with the cyclic order 
of the 4 branches of the curve. This can always be done in a unique way; 
an example is shown in Fig. 6. 


Figure 6: The Carter’s surface of this curve is a torus, which can be obtained 
by attaching disks along each of the two dotted lines. 


If n is the order of the Gauss diagram (the number of double points) and 
c is the number of faces attached, then the Euler characteristic of Carter’s 
curface is 2—2g =n —2n+e, hence g=(2+n-c)/2< (n+ 1)/2. 


Figure 7: Gauss diagrams of order 3. Out of the total number of 12 different 
Gauss diagrams of order three, 6 diagrams have genus zero, 5 diagrams are 
of genus one and 1 diagram of genus two. 
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3. Arnold’s basic invariants 


Let X be the space of all smooth immersions of the circle S! in the 
plane R?. Elements of X will be referred to as curves. 

According to a classical theorem due to H. Whitney [Wh], the space X 
is made up of a countable number of connected components each containing 
all curves with the same value i of rotation number. The rotation number, 
or Whitney index, of a curve is the degree of the mapping S! — S$! which 
sends every point in the direction of the tangent vector at this point, ie., 
the total number of revolutions made by the tangent vector when the point 
traverses the curve once in the positive direction. Since we identify the two 
immersions that differ by a mirror symmetry of the plane, we will suppose 
the Whitney index to be nonnegative. 

Choose a standard curve K; for every nonnegative value of 1: 


COD © GD GW 


Figure 8: Standard curves. 


Denote by + the set of all curves that have a point of direct self- 
tangency, i. e. a point which is visited at least twice with tangent vectors 
having the same direction (Fig. 9a). In a similar way, let ©- be the set of 
all curves that have a point of inverse self-tangency (Fig. 9b) and D%*, the 
set of all curves having at least one triple point. 


9 SK OX 
» <>< Oe 


Figure 9: Direct and inverse self-tangency. 


The discriminant is the union of the three varieties Mo Seeand 
Set 
C= Oe es oe 


An invariant of generic curves is a function on the space G which is 
constant on connected components, i.e., whose value may only change when 
we move through the discriminant hypersurface. Therefore the question 
initially raised by V. Arnold in [A]] is quite natural: are there any invariants 
of generic curves which have constant jumps on the discriminant? To state 
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the main theorem of [A1], we need to introduce a coorientation of each of 
the three components, 5+, U~ and D**, of the discriminant. 

If two curves are close to a generic point of U+ and lie on either side of 
this hypersurface, then they are identical everywhere but in a small disk, 
where one of them has two additional double points. We say that this curve 
is on the positive side of U*. 

The same definition also holds for the coorientation of the component 
a 

The fact that the triple point component is also coorientable is not 
so evident, and the corresponding definition is more tricky. We explain 
this construction in Fig. 10. It takes some imagination to understand that 
a triple point can be resolved in only two essentially different ways, 1.e., 
that the neighborhood of a curve with one triple point in the space of 
generic curves consists of two connected components. Either of these two 
components is given a sign + or — according to the following rule: 


Figure 10: Coorientation of °°. 


Consider the new-born triangle that has replaced the triple point. 
Choose an arbitrary direction on the curve. It supplies each of the three 
sides of the triangle with a certain direction. There is also a cyclic order 
in the set of the three sides defined by the order in which the sides are 
passed when moving along the curve in the choosen direction. This cyclic 
order induces another direction on each of the three sides. The sign of the 
new-born triangle is equal to 


(nee number of sides on which the two directions are the same_ 


This sign does not depend on the choice of the direction on the curve. 


Definition 2 The strangeness St is an invariant which jumps by 1 only at 
a positive crossing of the cooriented stratum &°*. 

J+ is an invariant which jumps by 2 only at a positive crossing of the 
cooriented stratum X*. 

J~ is an invariant which jumps by —2 only at a positive crossing of the 
cooriented stratum L~. 
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Theorem 2 (V. Arnold [A1]) For any sequence k = (ko,ki,ke,...) 
there are unique invariants Sty ; Jy ; Jy with normalization 


Ste (K,) =e, (A i, eee Ti aha 


Different choices of the normalizing sequence k may lead to additional 
remarkable properties of the invariants St, , Te ; Jy . In [A1] Arnold used 
the following normalization: 


for St: Ky Oy kj =i-1 (CS OR ae 
fot J 1 ck — OG ele) 
ee ee) ee rh Cee 


— and proved that it is the only normalization which leads to invariants, 
additive under the connected sum of curves. Below for the invariants with 
this normalization we will omit the subscript k. 

Another choice of normalization k gives invariants of spherical classes 
of curves (see Sec. 6). 

To calculate the invariants of a curve by definition, we connect the 
given curve with one of standard curves K; by a smooth path in the space of 
immersions that meets the discriminant only in its generic points. The value 
of the invariant on the standard curve is known. Accumulating all jumps 
of the invariant during deformation, we obtain the value of the invariant on 
the initial curve. 


Example 2. As an example, consider the plane curve 


Figure 11: An example curve. 


Figure 12 shows a deformation of this curve to K, and the correspond- 
ing jumps of the basic invariants. 

It follows that St. (T) =k +1, Ji (T) =k, + 2, Jy (CT) =k, —4. 

In particular, with the normalization we have adopted 
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Figure 12: Calculation of the invariants. 


4. Formulas of Viro and Shumakovich 


Choose an arbitrary orientation of the curve ' Cc IR2. Pick a connected 
component C of R?\T and a point z € C. 


Definition 3 The index indp(C) of the region C is the algebraic number 
of turns made by the radius-vector from x to a point travelling along I. 


Evidently, indp(C) does not depend on the choice of x € C but does 
depend on the orientation of I’. 

The index of the exterior component is zero. If C, and C2 are two 
adjacent components and Cj lies on the left of ! while C2 lies on the right, 
then indp(Cz) = indp(C1) + 1. We arrive at the Alexander rule? for the 
indices of four connected components in the vicinity of a double point of I 
(see Fig. 13). 


Figure 13: Alexander numbering. 


Now consider the smoothing of a double point determined by the ori- 
entation (Fig. 14). 


_ 
2J. Alexander [Al] was the first to notice that any generic plane curve admits such a 
distribution of indices. 
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Let I be obtained from I by smoothing all double points of I. It is a 
union of circles. Each smoothing joins two local components of the comple- 
ment of I’ with the same index into one component of the complement of I’. 
Therefore a connected component C of IR? \T’ inherits the index ind;(C). 


Figure 14: Smoothing. 


Theorem 3 (O. Viro [V]) 


Jt) = 1+n—} Jinds(C)x(C); 


C 
1— 5 ind? (C)x(C), 


Cc 


J“ (P) 


where n is the number of double points of and x is the Euler characteristic. 


This theorem can be proved by checking that the right hand side ex- 
pressions give correct values for the standard curves K; and verifying that 
their jumps under the three elementary perestroikas conform to the def- 
inition of the invariants. The same remark also holds for the formula of 
Shumakovich (Theorem 4) and formula of Polyak (Theorem 5). 


Example 3. Figure 15 shows the result of smoothing all the double 
points of our example curve. 


wa 
rar. 


Figure 15: Smoothed curve. 


FN 
ll 
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_ Numbers in this picture indicate the indices ind;(C) for all components 


C250, 


JT(L) 
t(D) 


(eo6 = (14) 2 
- (4a 


A. Shumakovich [Sh] gave a formula for the strangeness in similar terms. 

Fix a base point * on the (oriented) curve . Then each double point p 
of I obtains a sign s(p) = +1 or —1 depending on whether the first passage 
of this point starting from the base point * is positive or negative. (We 
have defined the sign of the passage earlier in Sec. 2). The signs of double 
points depend on the choice of the base point *. 


Definition 4 The index indr(p) of a double point p of I is the arithmetic 
mean of indices of the four components of the complement to I’ near p. (If 
one component approaches p twice from two opposite directions, then its 
index is counted twice.) 


Theorem 4 (A. Shumakovich [Sh]) Jf the base point * lies on the er- 
terior contour of T, then 


St(T) 5 s(p)indr(p). 


ae 


Example 4. We determine the signs and indices of all double points 
of our curve I: 


Figure 16: Indices and signs of double points of the example curve. 


= and findthat S({r)=1—-1+1— 1. 
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5. Formulas of Polyak 


M. Polyak in [Po] gave explicit formulas for Arnold’s invariants in terms 
of based Gauss diagrams, i. e. Gauss diagrams with a fixed base point *, or 
more exactly, with a distinguished arc on the outer circle of the diagram. 

The choice of the base point defines a sign s(c) of every chord c accord- 
ing to the rule: s(c) = s(p) where p is the double point that corresponds 
to the chord c and s(p) is the sign introduced in the previous section. In 
other words, the chord is positive with respect to the chosen base point 
*, if, starting from * and moving counterclockwise, its endpoint which is 
passed first corresponds to a positive passage of the double point. 

Let A be a based chord diagram. This is an object that differs from a 
based Gauss diagram in that the chords of A are non-oriented. 

A based chord diagram A defines a rational-valued function G ++ (A, G) 
on the set of based Gauss diagrams. In the definition that follows a subdi- 
agram of type A means a based Gauss subdiagram of G which becomes A 
after forgetting the orientation of all its chords. 


(4,6) =~ J] s(0), 


Ga cEGa, 


Definition 5 


where the sum is taken over all subdiagrams G4 of type A and the product 
is taken over all chords of G4. 


It turns out that, in the space of all generic curves with a fixed number 
of double points and a fixed Whitney index, Arnold’s basic invariants J re 
J~, St can be expressed in terms of the three functions (A,-), where A is 
a based chord diagram of order two. 


Theorem 5 (M. Polyak [Po]) Let G be a based Gauss diagram of a 
generic curve T. Then 


FS We Ceo ere = 
sy = (eat) aa) 


sr) = HL). +h). +3). 


ind*T — 1 
i n+ in 
4 
where n is the number of double points of T and indI is the rotation number 
of T. 
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Example 5. 
For our test curve 


Figure 17: Example curve with a base point. 


we have indr = —1, n = 6 and the based Gauss diagram looks as follows: 


Figure 18: Based Gauss diagram. 


We have three subdiagrams of type (): Zi : a : C/) : 
two of type ): = : es ; 
and ten of type (): A ; os ; cS ; ee : 
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(J ).0 = 1 = 1, 
(2,0) = -1+1=0, 


( ,G) = -141-—-1-14+1+4+1-1-141-1=-2. 
and 
Jt) = -146- 75" 22, 
Jao, ool 46-2" = 4; 
ob) (1/2) -14 SFI" a, 


6. Arnold’s inariants for spherical curves 


After adding one point “at infinity” the plane IR? becomes the sphere 
S?. Hence, a plane curve I can always be considered as a spherical curve. 
Since under this process different plane curves may become equivalent 
spherical curves, the natural question is: what plane curve invariants in 
fact represent invariants of spherical curves? 


In a recent paper [A3] Arnold noted that SJ*(I) = J+(L) + sind’? 
where indI° is the rotation number of I’, is a spherical invariant. Similarly 
SJ-(T)=J- (1) + ; ind’ and SSt(T) = Set) - zind?T are also spheri- 
cal invariants. Since the rotation number indT does not change under local 
perestroikas that consist in passing through D+, D-, DS*, the spherical in- 


variants SJ+, SJ~, SSt can be considered as usual plane curve invariants 
Jy , Jy, St, with a special normalization: 


-  9)\2 
meee meat f= — (i =1,2,3,...); 
= (i ~ 2)? . 
(Ones. = ky =r lar ae (ia 2a eee 
an 2 
ee 2 y= il ——— (i = 1,2,3,...). 


Example 6. The spherical invariants of our curve (Fig. 11) are 
SSULP) = 3/4; SJ*(L) = 5/2; SJ-(L) = —7/2. 
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A natural complete invariant of spherical curves is its Gauss diagram. 
So each spherical invariant in principle can be expressed in terms of Gauss 
diagram. Explicit formulas for SJ+, SJ~, SSt can be obtained from 
Polyak’s formulas by excluding the base point information. The follow- 
ing numbers take part in the answer. 


Definition 6 Let eS) be a Gauss subdiagram of a Gauss diagram G. The 


endpoints of its two chords divide the circle into four arcs. Let a be the 
number of endpoints of chords of G that belong to the upper arc, 1. e. lie 
between the two arrowheads of the oriented chords, and b, c, d are the 
numbers of endpoints on the other three arcs in the counterclockwise order. 
Then the weight w of the Gauss subdiagram is the number w @ = 


—a+b—c4d. 


Theorem 6 


SWenGn) 


—— - #0 +4Q+F dX He) 
OS) 

srr) = 24-4 +4 +F Me) 
Cs) 


sstr) = mat eC - 24-5 x 6 
&) 


where #) and a) are the numbers of Gauss subdiagrams of T of 
the indicated form. 


Example 7. It is easy to see from the example of Sec. 5 that for our 


favourite curve I: 
Q-. 1Q- 


and weights of the ten Gauss subdiagrams of type 0) are equal respectively 


to 
0, 4, 8 9O, 4, —4, 0, 4, 9, 8. 


So, ~2 "@- 24 and 
5 


3 
C1) = ep 
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Nonlinear integrable equations 


and nonlinear Fourier transform 


A. S. Fokas, I. M. Gelfand, M. V. Zyskin 


Introduction 


In this paper we study nonlocal functionals whose kernels are homogeneous 
generalized functions. We also use such functionals to solve the Korteweg-de 
Vries (KdV), the nonlinear Schrédinger (NLS) and the Davey-Stewartson (DS) 
equations. 

The solution of certain integrable equations in terms of formal power series 
was obtained in [4], [5]. In these papers the solution was expressed in a formal 
power series involving scattering data. In this paper in addition to developing 
techniques for multiplying and inverting nonlocal functionals we also: 

(a) Give the correct version of these series by giving meaning to the relevant 
kernels, see (2.10) and (3.18)). 

(b) We invert these series to obtain scattering data in terms of initial data. 

(c) Prove the convergence of these series. 

(d) We extend these results to equations in two space dimensions. 


1. Nonlocal analytic functionals withhomogeneous kernels 


The calculus of local functionals was developed by Gelfand and Dikii [4]. 
Local functionals of one function u(r) can be written as multiple integrals using 
the kernels given by the 6-function and its derivatives. For example, 


[e@e = [ eenuer) — 22) dx,dz2 , 
ox = J e@vuten)utes)e'(e — £1)6'(z — £2)6' (x — £3) drdx,dx2dzz , 


[ewrae = [ e@rutea)ules)u(a)u(as)6(@ — £1)6(x — £2)6'(x — 23) 
6'(a — 24)6'(a — 25) dadr,dr2... dzs . 


and so on. 
Nonlocal analytic functionals are those functionals whose kernels involve 


homogeneous generalized functions. 
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In the case of a real variable a basis in the space of homogeneous generalized 
functions is [1] 


ae a 


EO) "Gy 
In the case of a complex variable a basis in the space of homogeneous gen- 
eralized functions is 


hel 
-- (« +70). 


yee Te SET SED. 5 


6 — function and its derivatives 


Remarks. 
1) Only those functionals which make sense in the framework of generalized 
functions are allowed. For example, the functionals 


/ ies utes) S(t) dei, 


il 
ky + ko +120 


and 
1 1 


2 ———_—— 
fo Nera macy EY gy NE 


are not allowed, while the functional 
[re )u(k ee: Ca 
: g (ki + ke)? a 
3 Io° dk f°, dads (u? (25£) u (358) 


ve (54) -(t92) 0) 


is allowed. (see [1] for details). 
2) Local functionals are a particular case of functionals with homogeneous 


kernels, for example, 
i u*(x)dx = 
(xy i Nal 


= jf weryulen)o(e — £2) dx\dr2q = [edule 
Ne 


dx\dro. 


The product of two nonlocal analytic functionals is also a nonlocal analytic 
functional whose kernel is the direct product of the kernels of the two starting 
functionals. For example, 


u(ay)u(x (x1 — 2)” : (1 = yo)" ee alseee 
(/ (x1) u( )O4 1) (f wlan)utvadutes) Ta ry ee =) 


— 29)" (a3 — 24)" gh? 
=> [ ves)ules)ulesyulenulas) Ge Ra dx, a6 .daxs, 
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There are certain relations in the algebra of nonlocal analytic functionals. 


Examples. 


1) 

( i; uy (21)up(#2)@(1 — 21) O(a — 22)O(e2) d,dzs : 

(f estus)oa(vn)es(an)@(a ~ 20) ~ 92) ©Ca» ~ 25) OCs) ddande) 
= i Se aC ene en mica des 


= [ uy (x1)U2(x2)v1 (x3) v2 (4) v3 (25) (O12023O34O45 + 013032024045 
+0 13934042925 + 013034045952 + 031012924 O45 

+031 014942925 + 031914945952 + 034041912025 

+3404 015952 + 034045951012) dri...dxs 


ee i, eee) 

where (2) = aa = | @ 220° 

We have multiplied two functionals, with kernels of degree 0, and with in- 
tegration domain given by the simplexes, 0 < 2] < Z2 < land0< yy < y2< 
y3 < 1 ,respectively. For the product functional, the integration domain is not 
a simplex, but we can write it as sum of functionals, such that the integration 
domain of each functional is a simplex. The simplexes are given by all possible 
orderings of the letters 21,22, 41, Y2, Y3 such that 21 < Z2 and y; < ye < y3 for 
all the orderings (all shuffles of (x1%2)(y1y2¥s)). 


and 0); = O(a; — z;) 


Remark. The functional f ui(x1)u2(z2)O(y — 21) O(41 — £q)O(x2q) dx1dxr2 
with u(x) = <4, u(x) = 2 is the dilogarithm Lig(y). 


l-z’ 


Example 2. 
valor) ua(2) 15s) (24)us (5) 1232540 dx eta dx. 
— (| uy (x1)u2(x2)u3(x3)u4 (24) Us (5 )O12032 dx... ats) 


( ua(x4)uUs(Zs5) Os4dx4 = [ levator) eee us(®5)O12923034 O45 dx, an dx, 
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Example 3. 
1 dk, dk 
ul ean) 282) preg + ne) 
i dq: dqzdq3 
(f» (41 }¥2(q2)v3 (93) 7 oy; 4 qn 410) +qo+4q3)- Onis 


O(k k k dk, ...dks 
= fa (es)ua (ba) (hao aa he) GE ee) 


a [ale)us (ko)vi (k3)v2(ka)u3(ks)6(ki + ko + kg + kg + ks) 
- (p(1, 2, 3, 4) + p(1, 3, 2,4) + p(1, 3,4, 2) + p(1, 3, 4,5) + p(3, 1, 2, 4) 


dk, ...dks 


+7(8,1,4,2) + 7(3,1,4,5) + 7(8,4)1,2) + P03, 4,1,5) +7(8,4,5,1)) o=a5 


where 
p(m 72,73, ma) 


if 
(Kin, +10)(km + km +10)(Kny + kg + Kim + 10)(km, + km + kms + Km +10) 


Nonlocal analytic functionals appear naturally in nonlinear integrable equa- 
tions. For example, in the KdV equation the transformation from the potential 
to scattering data and the inverse transformation are given by nonlocal analytic 
functionals. We will write these functionals using the inverse scattering formal- 
ism [3]. Alternatively, one could start directly from the nonlinear equation. 


2. Nonlocal functionals for the KdV equation 


Let u(x) be a C™ real-valued function of a real variable x, with the fast 
decrease as x — too. 
We construct the following functionals of u : 


a(k) = jl + Le" [ ule2)u(as mies Uu(L2n)O12023034 Ce eae SN ye (1) 


6(x —2, +42 —...+ Lon) exp(2ikr) drdaxydr2...dron, 


1 ln 
b(k) = Di(k + 10) LO) sly [ u@yules) * 222° U(2on41)O12093 +... - Oanondi 
6(—@ +91 — 29+ 03 —...+ 2on41) exp(—2ikr) drdr,...dran, 


(2) 
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W(k, x) =1+ So(-) [ e@a)utes) Foo u(Zen)O(a1 = L)O21930 06.6.4 O2n,2n-1 
m=) 


6(29 — 41 + XQ —... + Len) exp(—2ikxo) dxodx,...dxon, 


(3) 


D(k, 2) =1+ ew [ e@autes) gn = u(Lon)O(z re £1 )O12023 osc op C2n-1,2n 
ot 
6(—x9 — 21 + Zo —... + Lon) exp(—2ikxro) drodx, ...dxon. 
(4) 


Remark. In the series, say, for U(k,xz) we can integrate over Toam41, 
m =0,1,..., to get another formula for U(k, x) : 


U(k,z)=1+> > a [e@du@) ... U(Lp—1)U(Ln) 
n=1 
O(zr, = L) 921932043 Borel Cnet 
(en Get?) — ems 22) —1)... (e~ 2#k(@n-1—-2n) =) dz,...dz,. 
We define S(k) by 
S(k) = (5) 


If |1 — a(k)| < 1, S(k) can be written as 


1 oo 
S(k) = FELD) » it u(xy)u(r3).--u(zen41) 


O21 923043 O45 - - - G2n,2n—-192n,2n41 (6) 
6(—2@ +21 —2%2+73—...+ Lon+1) exp(—2ikx) drdzxdx2...d£2n41 
Convergence. 


The series (1)-(4) converge for k # 0. They also converge at (oe, 
the moments of function u(z), that is, fu(z)dz, fru(x)dz, f x°u(x)dz, ...are 
small. The series (5) is convergent if (1) and (2) are convergent, and |1—a(k)| < 
Ne 
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Indeed, let k 4 0.Then 


1 
O(k,c)/St+ > —— | lu(zi)ule2)...u(te) 
sin k(x — #1) sink(2 — x2)... sin k(an_1 — tn) 


O(r = £1)O12003034 Coon” Ona dx \dzrq go6 dzn 
ae 4 1 (flute) 


For all k, including k = 0, 


|®(k,x)| <<1+ SS / |u(a1)u(ar2)... u(an)(a — 21)(@1 — £2)... (t4n-1 — 2n)| 
n=1 


-O(x = £1)Oj2 oon © one: dx dx oO o8 dan 


If the moments of function wu are small, the series is convergent for k = 0 as 
well. We will consider two operations on functionals (1)—(5): multiplication and 
inversion. ‘These operations are infinite-dimensional analogues of multiplication 
of functions and the inverse function. There could be some relations among the 
products of functionals. 


Example 1. 
Let us show that a(k)a(—k) — b(k)b(—k) = 1 for all k 40. Indeed, 


a(ka(—k) =1+ > S0(-)" / u(2t2)u(t4) ...u(:2n)@19@ 28... Com —1.29m 


pj ap =O) 
‘O2m+2,2m+1 * 92m+3,2m+2 --- O2n,on—167** 
6(—zo + 2 — %o+23- GASP oso = Lon) dxrodx,dz2 B06 dxon, 
co n—-1 
b(k)b(—k) 29) S>(-) )” f w(ea) Je(aa) .. -U(T2n)O12093-...- O2m,2m41 
n=l m=] 
©O2m+3,2m+2 * Q2m+4,2m43*--+* Son 2n—167** 
6(—29 +21—-—%4+73—-%4+...— Lan) dxodz\dzr Bete AL, 


a(k)a(—k) — b(&)(-k) — 1 = 7(-)" if Hence 


n=] 


exp(2ik(21 = 15) SIF aco = L2n))O23034 0 O15 veenees Oon 1,20 


Ghat ea ditior, (0), fy 24 (i), 
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From () one can see that a(k) = a(—k), b(k) = b(—k); therefore, |a(k)| > 1 


= _Jb)|_ 0) 
d |S(k)| a(k) ATTIC Og =< ke Oe 
Example 2. 


Let us show that U(—k, y)a(k) + U(k, y)b(k)e***¥ = ®(k,y) . Indeed, 


W(—k, ya(k) =1+ S(-)" / ene. GR 


- (012093034 -...- Oon—1,2n 


n—1 
a ie (O(x1 — y)©21 O32 -.-- + O2m,2m—192m41,2m42 
m=1 
-Oom+2,2m+3 * 92n—-1,2n) + O(x1 — y)O21932 ---- - O2n,2n-1) 
e~ 2*k20§(a9 +2, —22+...— Lon) dro... d£2n, 
also, 
W(k, y)b(k)e"Y = yey [enue -...+U(Z2n) 
ae | 
- (O(a, — y) 923034045 - --- - O2n-1,2n 
n—2 
+ 3 O(x1 — y)O21932 - .--* ©2m+1,2m * O2m-+2,2m43 
m=1 
-O2m43,2m+4°---* G2n—1,2n + O(21 — y)O21932- ---° Oon-1,2n) 
e~ 2*kt0 § (x9 + 21 — T2+...— Lan) daodx,...dZan, 
and 


U(—k, y)a(k) + Uk, y)b(k)e2**¥ = 14 SEY [ene eo (tan) 


n=1 
Oty a £1 )9 12923 eee Oeics 
6(a +21 — %2g —..-+ Zan-1 — Lon) dtodx,...d£an = (k, y). 


Example 3. 
Let us take S(k) ,given by the formal series (6), and a(k) , b(k) , given by 
the convergent series (1), (2). We can prove the following relation for the series 


in u(x): 
a(k)S(k) = b(k). 


The computation is similar to that of the Examples 1 and 2. 
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Inversion. 
Let us define S(x) = s(Ryert=S. We suppose that S(z) is a fast de- 


creasing function as x — +oo. Formula (6) can be written as 
d 
a *) =>— Ge SP J werules) 02 206(0 = 72) =p iy = x3)dxr,drodr3 
SPoac f u(x1)u(x3) see U(L2n+1)921923943 O45 see Onn 271 rn on el 
6(a@—2, +22 —243+...— 24) drydrq...dxon+1 +...) 


(here the right-hand side is a formal series in u(z) ). 
We can invert it, that is, we can express u(x) in terms of S(z) by the 


formal series u(x) = SJ W,(a), where W;(2x) is a nonlocal analytic functional of 


k=1 
degree k in S(x) (S(x) has degree one). The functionals W,(z) are determined 
recursively: 


Wi (2) = -+-S(e), 
W2(2) = — / ~-8(e1) = 8(25)0219205(z — 2) + 29 — 23) dxridrodr3 
24 dxr3 
= - [ $(21)5(es) (6(xq — 21) 6(x2 — 3)6(x — 2) + ro — 23) 
+6(22 — £1)0936'(2 — 21 + zo ~ £3) 
+0216(r2 — 23)6’ (a2 — 21 + 29 — 23) 
+0230936" (2 — 21 +22—23)) dridrodr3 
= = [ s(21)5(xs)6le. — £1)6(%2 — £3)6(x — 21 + 2 — 23) dx, drodr3 


= —9°(4) = 5, | Stex)S(@2)0¢@: — 2)6(x1 — xq) dx,dzo, 


(we integrated by parts), 


Walz) =—S> Sf Wang (21) Wma (29). Wine (a) 


k=2 my,mgQ...mp>1 
my+mo+...¢mpan 


©2193 > ©43045 Boo © On, 2k-1 92k, 2k41 


d(x = 28) ap (05) = 556 Sp aay, — ©2k+1) dx,dxro 665 dr2444. 
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Lemma. Consider the functionals of S(x), given by the formal series 


ce) — -£5(2) -- = [se)s@)0@ — £)6(x1 — 22) dx,dro 


=< > | stex)s@) .S(ton41)@(—a1 + 22) 


z 
a +29 —23+24)...O(—21 + 22 —... + Lan)O(—Zandi + ioe) ) 
O(—zen41 + an — Ton—1 + Lan—2) --. O(—Langi + Lan — --. + £2) 
6(a@ —21 +22 —23+...—Lan41) dridz2... dton+1 
a yf stexsea) eS (on) 
dz os 
O(-21 + r2)O(—21 + 22 — 3 + £4)...O(—21 + £2 —.-. + Zon-2) 
O(2; — x) O(a: —ao+23—2)...O(21 — 22+ 23 —--.+Zan-1 — £) 
6(2 —22+ 23 Pg. Pon) bi OL, 
He.) = (2) — 5 O-2) f $(es)S(a2) --- San) 
O(—27, + Se poo 104 es, O(a hy — ae te L2n) (8) 
Q(z, — y)O(z1 — 22 + 43 — y)--- (a1 — 2 + 3 —--- + Zant —y) 
6(a -—y+21 —22+23 Se tone) zi... droga 


O(—2; + £2)O(—41 + F2 — £3 + w4)O(—2, + 22 —--. + Lan—2) 
O(z, — y)O(z1 — Z2 + £3 Ey). Ole: =fo-+ 73 — --- ean =1 —y) 
6(x —%+22—-.-.-+ on) dx,dz2.. None 


We can substitute in these series S(x) as a functional of {u(x)} , given by 
the formal series 


Siz) [ee — x)u(z;)dx; + De [e@ules) ee Wwtro) 
es (9) 


@Qo1 O03 O43 O45 °---* O2n,2n—192n,2n41 


-6(z9 — 21 + 42-23 +---— Lon+1) dzodxy ...d£en41- 
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As a result of this substitution, we will have i(x) and U(x, y) given by formal 
series in {u(x)}. Moreover, u(x) = u(x) , 


3 OK: Ne 
UU iar) = Gi) [ener where U(k, y) ts given by (3). 
Proof 


We will prove the lemma by induction in the degree of {u(x)}. 
1) In the first order in {u(x)} 


Se) = i eo TG 

f(z). = Sey (2) = u(2), 

Hayley) = -(-2) f Sy(2)6(e-y +2) dr, 
See / SiGe eulogy eee 
=— / O(x1 — y)O(a#2 — 21)u(r2)6(x — y+ 21) dridro 
- / O(2, — y)O(x2 — 2 )u(xe)6(x — 2) + 22) 


-dz\dx2 = V,(1)(z, y). 


(In the last step we have made the change of variables x1 > y + x2 — Zi) 
2) Suppose that we have proved that 


N 
W(2,y) = 6(z) + S>(-)” [ wGca)uls) ... U(Lan)O(z1 — y)O21 932... O2n,2n-1 
n=) 
OF = 4-22 —...+2,) dtydra. .ano, O(N st), 
From the definition of U(x, y) 
G(a,y) = 6(x) — @(-2) f Ha,9)s¥u —2£—2}) dz. 
But S(z) is a series in {u(x)} with terms of degree > 1 in u, therefore, if we 


know W(z, y) as functionals of {u(x)} up to degree n, we can compute it in the 
next order (n + 1). 
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Notice that 
(—)” [autos Sa u(Lon)O(z1 oe y)921932943 O20 Con ene 1 


6(@ — 21 +29 —...Lon) dridrq...+ dron 


= (-)” | e@a)ues) one u(Lon)O(21 = y) 921932 O43 acme Oo2n,an—1 


6(e —y+21 — 2g +23 —...+ 2on-1) dx, dx2...dXon 


d 
= Sis | wea)ules) ... U(Lon)O(21 — y)O21 932943 -. - S2n,2n-192n41,2n 


6 (2 —y+a, — 294+ 23 —...+ Lon-1 — Lan + Zanqi) Aridaredz3...drondLon41 
(In the first step we have used the change of variables rz; — 22 + y — 41,43 
@o + £4 — 23,---,Lon—1 — Lon—2 + Lan — L2n-1)- 


d 
Also, 6(z) = is fom — y)6(x — y+ 2)dzx,, and 


N 
Bwaale,y) = — D> O(-2) i Tee Gia cohen 


m=0 


N 

d 

=~ 2 0-2) fF (ules)ulae) 062m) 
m=0 =D 

O(a, y1 921932943 - .. Q2m,2m—-192m+41,2m 

‘(29 —y+21 —%+ 273 —...+2om+1)) 


S(Nat omy © xo)dxodx,dx2...d£2m+1 


= @(-2) | u(es)u(ee)..-ulaans2) 


N 
(> (-)™41 0 (21 — y) 21932. 82m,2m—192m-+1,2m * (O2m+2,2m+1 + O2m+1,2m+2) 


m=0 


(G2m-+3,2m+2902m+3,2m+4 +++" Oon-+1,2N 2 +1,2"42)) 
-6(—2 +2) —%2+273—...+22an41 — LQN+2) dx ,dxq...dton+2 
= (-)¥t1@(-2) [ues ...u(zon+2) 
(O(x1 — y1)©21932043 ----- OoN+2,2N+1 — ei 
- O(21 — y)O12932O34954---- @oNn41,2N 92N+41,2N+42) 


6(-x +2, —%9 +273 —...+ 22N41 — 29N+2) dx dz2...22N+2 


150 A. S. Fokas, I. M. Gelfand, M. V. Zyskin 


The second term in the last expression is zero, because the volume of the 
integration domain vanishes ; the first term coincides with the term of degree 
(N +1) in (az, y), see (3). 

d 
To prove the formula for u(r) we use the relation u(x) = am i 
x 
S(x - 1) V (a, x)dz1, which follows from the definition of % and ¥. We know 
both S(x) and W(z,y) as functionals in {u(z)}. The calculation of the same 
type as above gives that only the first order term in {u(x)} is not zero: 


: da in 

u(x) = = dO) jf e2)ula) ..- U(Lan)6(—21 + Zo —... + Zen) 
(O(a, — ©)O219320 43... O2n2n—1 — (—)” 

O(21 — £)O12O32O34954 +... - Oon—1,2n-292n-1,2n) 

Ogjdtg. dio — ue) 


The formula (7) can be written as follows: 


+ ky + 10)(Ko + kg + 10)... (Kn—a + kn +40) 
dkdk,dko...dkp 
(Qi2)” 


Tae S(k1)S(k2)...S(kn)(ki + ke +... + kn) 
(x) DU y 


exp(2ikx)6(k — ky — ky —...—kn) 


Any polynomial in u(x) and its derivative can be written as the functional 
of the same type, but with some polynomial in {k;} in the numerator. This 
property is similar to the usual Fourier transform of the linear function of u and 
its derivatives . Therefore it is natural to call transformation (10) the Nonlinear 
Fourier transform of u : 


yer) a, (42) ee yam) = Am (2i)drtdat Adm 


Sa / S(ki)S (kz)... S(kn) 
= (ky +ko+ 10)(k2 +k3+ 10) rae (aes + ky+ 10) 
exp(2ikz) -6(k —ky —kg-...- kn) - SYM4, dp...dm, 


se (ky Fear cc seh ihe (es, + kp, 41) 
1<p1 <p2<...<pm-1<n 


(kp. 41 ae kp, +2 Gipes < ote (Pe PERS, a Kp +1)(Kpo41 Ar Rp +2 ae ie eam 


dn, 41 dkdky dk... dhen 


(Kp, at Resa) (Kee ap Kone +...4+ kn) == 
(277) 


(11) 
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Examples. 
Guu, = 32i > / igs hi) 5h) S ae 
= (ky + ko+ 10) (ke + k3+ i0) Hae (Kyat + ky + 10) 
exp(2ikz) -6(k — ki — ka —...— kp) 
dk dk, dkz2...dkn 
Qi 
— S(k,)5(k2)S(k3)... S(kn) 
User + Guu, = —32 / Se ae eee 
op (ky + ke + 10)(ko + kg +10)... (kn-1 + kn + 10) 
exp(2ikxr) - 6(k — ky — kg —...— kn) 


+k+ ((ka + ko +... 4+ kn)? — (ki? + ko? +... + kn*)) 


dkdkdkp ...dk 
+k - (kr? + ko? +... + kn’) = a 

(12) 
We see that for the differential polynomial uzz, + 6uuz the corresponding 
polynomial in {k} after the nonlinear Fourier transform is (ky +kee +...+ kn), 
n= 1,2,.... We can use this fact to solve some nonlinear equation. Let S(k,t) = 
So(k)e8****, and u(z, t) is defined as a functional of S(k,t) by (10) (with S(k, t) 

instead of S(k) ). Such u(x,t) solves the KdV equation 
3 


d o Ou 
— ult) => Aza ule t) + 6u— 


Ox 
Indeed, for such u(z, t) the right-hand side of the equation is expressed in terms 
d 
of S(k,t) by (12), and it coincides with — Gul, t)- 
Let us find polynomials in u and derivatives of u such that after the Nonlinear 
Fourier Transform the corresponding polynomial in k is given by ) hae : 


3 


| =1,2,... .In order to do this, let us first compute the resolvent: 


= S(k1)S(k2)S (ka) --- S(kn) 
ce ee re} (b+ ki + 40)(K + kg + 20)... (Rn—1 + kn + 10) 


dk, ...dk 
exp 2i(ky +... + kn) —-—— 
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R(k, 2) = U(k,z)U(—k, 2 — 0) 


= £3) [ 5(61)5(h)..-S(bn) exPQilhy +... + bs)2) 
n=1 

n-1 

& 


m=1 
pn Se 5 A ek SEE 
(Ri tha +10) (Ratha +40). mina thm 440) Ke, FR+i0)( Keita —k400)(Em4i thn 4+2400)...(Rn—1+hn+i0) 
a il 

(k +ky+ 10) (ky +ko+ i0) Bas (keen tknt+ i0) 


a 1 ) GN co cA. 
(-k +kyt+ 10) (ky t+ko+ i0) ie (Raat tky»t i0) Cae 


aoe | S(k1)S(ka) ...S (kn) exp(2i(ki + ..- kn) 2) 
(= OG eee ahs ne cate + 10) 
= 2 1 és i " 1 7 1 dk, ...dk 
4 km FRI kimy K+)" k+k, +00 —k+h +10) (mi) 
S(k1)S(ke)... S(kn) 
os) ooo cee 
” | CEU e a a ee 


= i dk, ...dk 
por om 1 ee 1 kas 1 1 n ‘ 
ye po ee rom 


~ 


S(k1)S(ko)...S(kn)k exp(2ikz) 
(ky + ky + i0)(ko + kg + 10)... (kn-1 + kn +10) 
dkdk, ...dk 
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1 
——_——— 2 " 
3 ie [ou +u') 
=> / S(k1)...S(Kkn) (ka? + ko? +... + Rn*) 
(ki + ko + 10) (ke +k3+ 10) ae (Rye +k,+ i0) 


dk,...dky 


ahi +--+ hn) oan 


(10u? + 10uu” + 5(u’)? + ul”) 


ai G(r) olkki ko ee) 
(ky +ko+ 10) (ke +k3+ a0). sienee +ky+ i0) 


2) let S(k,t) = S(k)e*"*,1 = 0,1,2,..., and 


i R! aeemreenee eae 
m2 £4 J (ky + kp +10) (ko + kg + 10)... (Rn—1 + kn + 10) 


CO a aS (ky +... + kn) ie 
Such functionals I,m are conserved: Slim = (0). 
Theorem 2.1 (Solution of the KdV equation). 
Consider the Cauchy problem for the KdV equation 
me Bi ule B= ca 9,3 ule t)+ euse, 120 1G, 0) a) (13) 


Solution in Pe power series. 


Oe “13 S(k1, t)S(ka,t) --- S(kns t)(i +--+ + kn) 
(ky + ko+ i0) (ke +k3+ aOjee Urea +kyp+ i0) 


A alee O45 


exp(2i(ki +... + kn)x) (Omar 


where S(k,t) = S(k) exp(8k*t) 
1 co 
S(k) = ~ Di(k + 10) % | wenules) O6 pu Doe) 
O21O23043O45 ----- G2n,2n-192n,2nt1 


6(—2 +21 —%o+%3—-..-+ Lon41)exp(—2ikax) drdx,...den41 


u(x) = u(z, 0) 
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is the solution of the Cauchy problem. 
Solution in convergent series. 
b(k 
1) Starting from u(x) define S(k), {ikn}*_,, {en}4_, as follows: S(k) = mat 
k € R, where 
a(k) =1+ S-(-)” [ wa)ulcs) ...U(Len) exp(+2ikz9) 


nd 
6(x0 —4b)) + 1B) — Bao + L2n)O12923 ao9 elon 


dzodz,...dton, ke Ct 


b(k) = Se Bo [e@)ucer) ee u(Xen41) exp(—2ikzo) 
6(—@9 + 21 — Fo +23 —... + Lon41)O12O 23... Qan2en41 


dxodx, O08 dron+1 


S(k) = S(—k);|S(k)| < 1,k #0 
Ce oe Kn € R* is the set of zeroes of the function a(k),k € C* 
Cn 


Cn = =—————, where 
Aull) ae 
Se DO, O(iK,,,0 , 
en = ofr? O(ikn, 2) = BES) (the ratio does not depend on z) 


Wikn,z)  — V(ikn,0) 
(k, xz) and W(k, x) defined in (3), (4). 

Define ap 
SG.e i S(k) exp(2ike + 8ik*t) 


Ga(t) =e, exp( x,t) 


For the class of initial data u(x) such that the function S(z) is of fast decrease 
as x —> +00, the solution could be written as follows: 
Define the Fredholm determinant and the 1st minor to be 
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Dee = 
oo 1 loo} 
1+ y = jh ©(—y1)O(—y2) aol l(WAle ws eee — yi — yj, t)|dyidyz...dyn 
n=1 =e) 8 


0(-w)S(e-y-w.t)+ > 0(-v) | O10)... O(-an) 
n=1 — 


S(e@—y—yo,t) S@—-y-y,t) ... S@—y—yn,t) 
S(@—y%—yo.t) S@—-yi-m,t) ... S@—y — yn?) 

- dy dy2..-dyn 
| S(x — yn — yo, t) S(z — yn — yi, t) see S(z — Yn — Yn) t) 


d ae (3) oP 
OG) —= ear S(x —y,t) | d(y) — a an dy — 2572 In det A(z, t) 


° —(Kntkm)£ 
deo te ee 


(Kn + Km) 
Di (%) 
(jer: | Mere ne eu O(-y1 )dyody1. 
at 


This solution is defined for the class of initial data such that the Fredholm 
determinant and 1st minor are convergent.For convergence it is enough to have 
S(x) of fast decrease as « — +00. It can be proved that the Fredholm determi- 
nant is not zero. 


3. Nonlocal transformations for the nonlinear Schrodinger equation 
(defocusing case) 
Let g(x) be a C® complex-valued function of a real variable x, with fast 


decrease as x — +oo (Schwarz class). 
From q(z) construct the following series: 


a(x) = 6(x) + _ [. G(21)9q(v2)G(a3) -.. q(L2n)O21932043 -- - Q2n,2n-1 


6(x +21 — 22 +23 —...+ Zen) dz ,d2zq...dXan 


(1) 
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W(x) = » | q(£1)4(2)q(a3) .- - 9(T2n)4(Ln41)O21 93243 *--- > O2n41,2n 
m=O % eo 


6(@ — 41 +22 —-...4+ Len — Zon41) dxydro...dxon41 


(2) 


(2,9) = ste) + >> ” aaia@esu(as). aan eG = rer Onuee am 


O(a — 2%, +22 —23+...—fan—1 + Zon) dr,...dron 


(3) 


2(z,y) = >> i q(21)a(x2)4(@3)...4(t2n)G(t2n41)O(y — £1) 1223. . O2n,an-44 
7 — 0 mcs 


6(x = WS = £5 FP seo = Lon + Lon41) dz, oe d2on41 


(4) 


oo 


a q(21)9(z2)9(x3) ... G(t2n)9(r2n41)O(z1 — y)O21 O39... Oon4+1 


Vi(z,y) = ->f 


0 
O(a+y— 2} ip ge ee. ye) CEje. Olay 


(5) 


Vo(z,y) = 6(z) + yy [- (x1 )9q(x2)G(x3) ...4(r2n)O(a21 — y)O1 Oa0... ©G2n,2n-1 


6(2+21 —2o+273—...4+29n-1 — Zon) dry... drton 
(6) 
The integration domain, say for nth term in W,(z,y), is the intersection of 
the region y < 7] < Zo < ... < Lon41 With the hyperplane z+ y — 21 + zo — 
Ber ooo = Mpa =O. 
Lemma. The series (1)-(6) are convergent. 


Consider, for example, the series for ©, (x,y). Let Q = max|q(z)|. 


| f aes aea)aCes) -+- G(an)O(y — £1)O12903 -...- Oon—1,2n 


6(z —2, +. 29 —...+ fon) dz,...dxron 


= 0- fl la(z1 )lla(z2)|. -. |¢(t2n-1)|O129 23... S2n—1,2n dt, ...dton—-1 
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Define S(x) = il eS where 
a T 


b(k) = i biG ome ada c(h) — / a(x)e~*** dar (7) 


Lemma. 
1).For q(x) such that |1 — a(k)| < 1 S(2) is given by the convergent series 


(a) = (9 f a(er)alca)ales) .. a(@on)a(oants) 


n=0 
012932034954 -.-- - Qan—1,2nG2n41,2n (8) 
-6(x —%, +240 —-...+ Lan — on+1) dx dx... dXon+1 


2).Consider S(x) ,given by the formal series (8). We can prove the following 
relation: 


J seenae — 2,)dz, = 0(z). 
Here a(x) and b(x) are functionals in q(x), g(x), given by the series (1) and (2). 


Indeed, let us collect all the terms of the same order in q, ¢ in the convolution 
of the series (1) and (8) 


[sac — y)dy = jf ensey — 21)6(x — y)dx,dy + | a(es)ale2)a(zs) 
(—©129326(y = Hi ae i) — x3)6(x = y)+ 


O326(y — £1)6(x — y + £2 — £3) dx dx ydxr3dy 


++ f a(es)aten) alton 
(O12O32O34954 -. - O2n—1,2n—292n-1,2n G2n41,2n 
n-1 

oi Se (—)—" © 12032934054 - - - Q2m—1,2mO2m+1,2m 
m=0 


(Q2m+2,2m+1 + O2m+41,2m+2)O2m+3,2m+2 


* O2m+4,2m4+3 oa oMe Oren) 6(x — 21 ap ain, — Hee) Sao Done) dz, B00 dzan 


[oe] 

= Y, | a(er)dle2) -..4(ean+s)On1On0 as +... Oan41,20 
n=0 
(2 — 21 429 —23+...— Zen41) dz,...dXon 


== (2) 
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(We used oF SF oe = 1). 


The convolutions of functionals (1)—(8) are again the functionals of the same 
type. There are certain relations for the convolutions: 


[aerate +2)dz, = 6(x) + eve +21) dx, (9) 


J ler Pale— 21 +9) ~ a(es)B(-2 +21,y)) der = —Wole,y) (10) 


/ (b(21)®1 (2 — 21 + y, y) — a(x1)$2(—2 +2, y)) dx =-Vy(z,y) (11) 


Let us prove (10). We have to collect all the terms of the same degree in q, 
q in the left-hand side and to compare with the right-hand side. 


i) (®2(x — 8 +-y,y)0(s) — ,(—2 + s,y)a(s)) ds 


=-6(2) + f qer)a(va) (Oly - 21)6(e — s + 21)8(s — 22) — Oly ~ 21)Or 
d(x —s +21 — £2)d(s) - —Oo16(z — s)6(s + 21 — £2)) dxidrods+... 
+ f a(cs)a(es)aes) ...4(22) 
n—l1 
; (= O(y — £1)O12923934 - ... - G2m,2m41(O2mi1,2m+2 + C2m+2,2m41) 
m=0 


O2m+3,2m4+292m44,2m+3 *-+-* Oan,an—1 — 921932 -.--* Q2n2n-1 


— $5 Oly — 21)@ 12023 - ... - O2m—1,2m(C2m,am41 + O2m-+1,2m) 


Ti 


©O2m+2,2m+1 * G2m+43,2m42+---* G2n,2n-1 


= O(y = L1)912093 ao Oon,2n-1) O(@ +%,—-—Zo9+...-— Lon) dz,...dran, 


= -6(2) - 3 / G(01)q(@2)4(ws)... 4(at2n)O(1 — y)@21O a9 -..-- Gan on—t 
a 


o(z +21—- 8) SP 505 = Lon)dxy ae don = —WVo(z, y) 


The proof of the other relations is similar. 
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In addition to convolution of two functionals, there is another operation for 
our functionals , namely inversion.It is the infinite-dimensional analoque of the 
inverse function. Consider the series (8) 


S(z) = s Q(2i41)(Z) 7= 


7=0 
= q(x) - J a(es)a(ea)a(es)@120s06(2 — 2; + 22 — 23) 


+ f a(c1)a(v2)q(@s)4(e4)a(a%5)©120s20s10ss 
6(2@ —2, +%9-—23+24—25)—... 


S(x) is a formal series , its nth term to is a nonlocal analytic functional of 
q(x) , G(x) of degree (2n + 1) in q,q. It can be inverted, namely, g(x) can be 
expressed in terms of S(z): 


q(z) = S(1)(z) + S(ay (x) + Sis)(z) + --- 
where Sim(x) is a nonlocal analytic functional of S(z), 5(x) of degree m in S, S. 
Say(z) = S(2) 
S(3)(x) = [ Ste) 5@2)5(es)6@ — 21+ 22 —23)912032 dr1dr2dx3 
S(s)(2) = — [ (21)5(e2)S(2s)3(v4)S (es) C120 120 Ore 
6(a@ —21 + 22—-23+ 24 — 25) dz,...d&5 
+ / (si) (a1) 5(x2)S(a3) + 5(21) 
5® (x2) S(w3) + S(e1)5(a2)S§) (z3)) 


(3) 
0129326(x — 455) ap iy = £3) dx idxodxr3 


= J Se)5(@2)S(es)5(e4) (es) 6@ —2,+22—2%3+ 24-25) 
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( —912932034954 
+©32032  =954O(21 — 2g + £3 — 24) 
023043 «= O(a@1 — Fo +. 23 — Z4)O(25 — 24 + 23 — £2) 


+ O49 034905490 (z5 —%+2%3- X2)) dz,...dzr5 


= [Se 5e1)S(2)5 es) 5(es)6te —2%1+22—23+24—2s5) 
O120(21 — 22 + 23 - £4)9O540(@5 — 24 +23 — £2) 


(—O32O34 + O32 + 923043 + O34) da,... drs 


= J 8(e1)5(22)5(22)5(e4)5(25)0 200 —@24+ 23 —2%4)O54 
‘O(@5 —%4 +23 —£2)6(e-—21+...—25) dr... das 


(Indeed, —O32034 +©32+ 023943 +034 = —O32O34+O32(O34+O43)+O03043+ 
O34 = (923 + O32)O43 + O34 = O43 + O34 = 1. 


Lemma. Consider the nonlocal analytic functionals of {S(x)}, given by 
formal series 


q(x) = S(x) + > / (S(21)S(x2)S(es) ... 5(v2n)S(t2n41) 
O(r, — Se. =<) 41-03 = 2). .-O(¢) 25 + 7.2 oy) 
O(en41 — Lon)O(Lon41 — Lan + Lan-1 — L2n2)-. -O(f2n4+1 — Len +... — £9) 


6(a@ —2, +29 —...4+ fon — Doe) ate Oo. 1 
(12) 


81 (c, y) = 6(2) + x i (S(1)5(w2)S(#s)... 3(Con) 
Ele Ge +%2—23)...O(y—21 +... —2an_1) 


13 
-O(x1 — £2)O(a1 — ro + 23 — 24) -...* O(a) — 29 +...—Zon) - 
6(m@ — 21 +2%2—-23+...—2on41 +n) dz,...dzan 
b2(2,y) = » , | $(2)8(e2)5(2)...8(@an)5leanes) 
Ee ie O(y — 4, +22—23)-. -O(y— 21+ 22-234... ~- 22-1) 
‘O(z1 — 22)O(a21 — ro + £3 — 24) -...°O(t1 — 22 +...—22n) 
6(2—y-to—g+... SHE ire) GH a oeeny aie 
(14) 


Let us substitute in these series S(x) as formal series in {q(x)}, (8). The 
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result of such substitution would be formal series in {q(x)}, and , moreover, 
q(x) = (2) 
$1 (x,y) = 1(z,y) 
| bo(x,y) = Bo(z, y) 


as formal series in q(x) , with ®,(x,y) and (x,y) given by (3) , (4). 
The series (9) could be used to get the solution of a nonlinear equation. In 


order to see this, let us rewrite (9) in terms of S(k) = ii S(z)e~?*** da and 


—oco 


Ee (.) a S(x)e**** de : 


(«) = Oaks Sa) Sf)» Some) 
: 2a J_., (ka — hi + 10)(Ka — ke — 10) (a — Bey + 10)... (ana — kan — 10) 
oy cle 
-exp(2i(ky — ko + kg — ... + kon41)2) So 


(15) 
OO 

The kernels =a = i ©(x)exp(+2ikx) appeared as the Fourier trans- 
U —0o 


form of the Heaviside function kernels. 

The series (15) has the following property: a polynomial in g(z) , q(x) and 
their derivatives can also be written in the form (15) but with some polynomial 
in {k} in the numerator: 


co 


d”q(z) 
=2- (2i)” 
dz” ee) 2, 
/ S(k1)S(ko)....S(kon41)(ki — ke +--+ kon + kon+1)” 
(ko — ky + 10)(kg — ko — i0)(ka — kg + 20)... (kon41 — kon — 10) 
dk, ...dkon41 
-exp(2i(ki — ke + kg —... + kon41)2) See 


(16) 
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S(k,)5(k2)S(ks) Sees S(kon)(—ky +kg—-...+ kan) 


i (ko —kyt+ 10) (kg —kg- 10)(ka —k3g+ 10) sis (kon-1 — kon_-2 - 10)(kan — kon 1 + 20) 


dk, ...dkan 
fexp (2c nie ee ko 


(2) 
q(x)q(x)q(z) = 45> 
aa 
i S(k1)5(ko)S(k3) ... 5(kon)S(kon41) 


(ko — ky + i0)(k3 — ky — 10)... (Kon — Kon—1 — 10)(Kong1 — Kan + 10) 


. (cs —ko+...—ken41)? — Wa + ys tae] 
= p=0 


pl 
dk, ...dkon 
exp(2i(ky — ko +...— kon + kony1)) —3 


(Qm)je742 
g1(a)gn(a) = 2241 > 
/ S(k1)S(k2)S(k3) aoe S(kon)S(kon+1) 
(kg — ky — 10)(k3 — ko +70)... (kon — kon—1 — 10)(Kon41 — kan + 10) 
3 (—k) + ko —...+ kop,) 
1<pi <p2<...<pm<n 
(Kopi +1 — Kop, )(—Kopy+1 + kop, 42 — ++» + Kop, )(Kopo41 — kop) +. 
dk, ...dkan+1 
(Fane, +1 + Kap, +2 — --- + Kap) (Kop +1 — kopen) One 


(17) 
The fact that both differentiation of g(x) and nonlinearity under the “nonlin- 


ear Fourier transformation” (15) has the same effect, namely, some polynomial 
in {k} appears in the numerator, can be used to solve nonlinear equations. 


Theorem 3.1 (solution of the NLS Equation). 
Consider the Cauchy problem for the NLS equation: 


6 0? 5 1, defocusing case 
iggatest) + zaalest)—2Aela=0, A={ _p Sefocusing et 


(18) 
q(x, 0) = q(z) 
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(Solution in formal series). 
The solution of Cauchy problem is given by 


=~ S(k1,t)S (ka, t) ... S(ken41,t) 
t=250" 
a(z,?) Be: i (Ea — ky + 40)(k3 — ka — 0)... (ka — kg + 10)(Konti — Fon — 10) 
exp(2i(k) — ko +...—kon + kon41) 


Ge cco 
(Qa2et 
(19) 
S(k,t) = e~**"*S(k) 
S(k) = yom | q(@1)4(22)q(#3) --- @(t2n)4(t2n41) 
n=0 Se!) 
©12932034954 -- - Q2n—1,2nO2n41,2n 
exp(—2ikax)6(2 — 21 +22 —..-+ Lan — Lon41) dadx\dr2...d£2n+41 
(20) 


(Solution in convergent series, defocusing case \ = 1). 


y 
(zt) = [ Se- nd) eee dy 


" 


S(z,t) = i a exp(2ikx — aint), b(k) and a(k) are defined by (1) (2) 


me (“) Leyva = 


pi 
n! 


n=1 
@(-u) / 2 iain ene 


Kyrt(y,yo) Ketlyyi)  --- Kz,t(y, yn) 


eee Le ,9T 
Kz,t(y1, yo) Key) (Yt Yn) dy; ...dYn 


Kz 2(Yn; yo) Kat(Yns¥1) cee GS Vey) 
De =1+ Pf ou ye(-te).--O(-on) [Kealusras) hs don --- 
(pal ‘ 


Kealvrsye) = f O(-v)5(@ — 11 ~V)S(@ —y—8) dy 


164 A. 8. Fokas, I. M. Gelfand, M. V. Zyskin 


The solution is defined for the class of initial data q(x) such that the Fred- 
holm determinant D, and the first minor Dz, (@) are convergent. It can be 
proved that the determinant is not zero. 


Proof (formal series). 
2 


O 
1) We compute Bq a(t t) and |q|?q in the same way as before, see (16) ; 
2 


Bs eee OE 
iz a, t) + sas, ql’q 


oe) yas els 
2 s = 2 2iky £ 1 
= fe ay th?) S(k, the* 


(oe) 
6) 
DG Scar alee cape rae 2 
ee (iq, — Ahi” — hp” + + kon417)) 
S(ki,t)S(ko,t)... S(kon+1,t) 
(ka —kyt+ 10) (kg —ky- i0) ae (Ken+1 — kon — i0) 
dkiendi oa 


Pi exp(2i(ky — ko + k3 = oan Se kon41)2) (Q7)2n+1 


=0 
for S(k, t) = e~4*"*S(k, 0) . 
2) The substitution of S(k,0) = S(k) as series in {q(x)} (see (19)) into (18) 


gives q(x,0) = q(x) (see Lemma) . 


4. Nonlinear transformations for Davey-Stewardson equation 


Let q(x, y) be a complex-valued C™ function on the plane R?, with fast decrease 
as |x|? + |yl? > oo. 
We construct the following nonlocal analytic functionals of {q}: [4] 


a (oe) 
aie) 5 
n=0 
/ q(21, 21) (22, 22)q(23, 23) --. (Zon, Zon)q(Z2n415 Z2n41) 
(21 — Z2)(22 — 23)(23 — Za)... (Zak — 22841) (1) 
-exp (k(Z —Zo9+2Z3—... a oh) = k(z = 2) Se FB = 3 SE 22k+1)) 


d?z,d?zo see @ Zona 
ge as 
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[oa] 
palk, k; 2, z) = lap SS 
Ie 
/ (21, 21)9(Z2, 22) .-- F(Zen—1, Zen—1)G(Zen, Zan) 
(e = 2)(Z — Z2)(Z2 = z3) alee (Zant = Zon) 


-exp (k(z = 22 + 23 Sele + Z2n41 = Zon) = k(21 = + PRU oo + Z2n+1 7 z2n)) 
d*z; AO ae 
(Qi)\2 
. (2) 
bo(k, k; z, Z) = Se 
n=0 


(21, 21) 9q(Z2, Ze)... F(Zen41; Z2n+1) 
(Z — 21)(2z1 — 22) (Zo — 23)... (Zon—1 — Z2n)(Zan — Zn) 


-exp (K(Z — 2 + Zo — -.-+ Zon — Zant) — b(2z — 21 +29 — + + Zan — 22n41)) 


d?z, aes d?zon 
(Qn) 
(3) 
where 2 ay ieee = 2 — 129, kh = ka the, & — Ki — tho, d2z:= ~dzdz. 
The series are convergent for some class of functions g. We will not investi- 
gate convergence; we will work with these functionals as with formal series. Each 
term of these series is an integral, involving homogeneous generalized functions 


i as kernels ({1 ]). 
e 
There are the following relations for the functionals (1 )-(3 ): 


es = ij = 
712 (4, K, 2, Z) = kye(k, Ky2, Z) a gal Z)ur(k, k, Z,2)) 


On x eh2-k 


Bk me 
0 Re 
a = € = Oink 


The series (1 ) could be inverted, that is, q(z,Z) could be written as a 
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functional of {a(k, k)}: 


a(ksk) = f ater, i)ex (hes — hax) SE + f (Meets alates) 


= d? 2, d2 2z9d2z 
exp(k(Z; — Z)+ 23) — k(z = By SP 2z3)) ) Oxi" a + 


G2, 2)— a1) (z, Z)+ Q(3) (G2)... 
io = Se elk 
Q@qa)(z,Z) = =2 f ak, F:) exp(—k, Z+ kz) “7 


Sean _2 ou(ky , ky )&(ke, ka )a(k3, ks) 
a3)(z, Z) —_ mn? i (z = 29) (Ze _ 23) 


exp ((—k1 2 + kez — kazg + k(% — Zo + 23) — kz) 
d? zd? 29d? z3d7kyd2kod2k3d2k 


2 f aki, ki)a(ke, ke )a(ks, ks) 
98 fee OA tk) 


exp(—k2 + kz)d?k, d2kgd?k3d2hkg 


=2 f Seu flatts Relate, Bs) 
(ko — ky) (k3 — ke) 
dk, d2 kad? kg 
aT 


exp(2(—ki + ky — keg) — z(—ky + ka — kg) : 


Lemma. Consider the following functionals of Lak, be) 


co 
jig (k, haze) = 1+ 


n=i 
/ _ = o(ki, 1 )e(ke, ko) te a(ken-1; kon—1)@(ken, kan) 
(ke = ky) (ks gx ka) (ka x k3) see (kon-1; kon—2)(kon ae Kon—1)(k se kon) 
a exp(2(—ky ae ko = k3 SP ooo = [Piyesa ae kon) 
Os spa 
—2(—ky + kp — kg +... — Kon—1 + kon) —— 
(4) 
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fig(k, he Z, Z) = ye 
n=0 
i a(R, k1)a(ke, ke) ..  O(kan—1; kan—1)o(kon; kon)&(kon-+1; kon+1) 
(ka — ki)(k3 — ko)(ka — kg)... (Kan—1 — Kon—2) (kon — kon—-1) (Fanti — kon) (k — kan+1) 


exp(2(ky _— ko +... kon+1) _ 2(ki —ko+tkg—...+ Kone) 


d*ky shake a kon+1 
qentl 


(5) 


q(z, z) =-2 Se 


n=0 
ar(ky, ki )@(ke, ka) --- (han) kon+t) 
(ko — ki) (kg — ke) (ka — k3)--- (Kon+a — Kan) 


: exp(Z(—hy ap ko = 005R kon = kon+1) = z(—ky +kg—...+ kon - kon+1)) 


d-ky ...d°hpeey 
qentl a 
: (6) 
We can substitute in (4)-(6) a(k,k) as a functional in q(z,2Z), given by (1). 
As a result of this substitution we will obtain functionals in {q(z,Z)} ,given by 
formal series. Moreover, 


jiu (k, k, z, 2) oa pikes he 2,2) 
[in hokees2) = Halk ky 242) 
q(z, 2) = q(2,2) 


Theorem 4.1 ( Solution of the Davey-Stewartson equation-II ) 
Consider the DS equation 


iP g(2, 2,1) = —(0 + P)alz,2,8) 
1 i a 
+ hale, 2,1)(510+ 92) lal,2,9P) 


q(z, 2,0) = 4(z,2) 
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The solution is 


oo) 
Giz, 2,1) —=—2 ye 
=) 
a(ki, ki, t)a(ke, ka, t) oo8 a(ken+1; Kant t) 
(Fi — ka) (ka — a)(ha — Fa). (Kan — Rona) 
-exp(Z(—ky + ko = os05F in — kon+1) = 2(—ky +kg—...+tkan — kon41)) 


d*k,...d*ky 
qent+l 
(7) 
oO 
a(k, kh) — >y 
7) 
‘i Q(21 21 )G(2Z2, 22)q(23, 23)... G(Z2n, Zen) q(Zent1; Zon+1) 
(21 - Z2)(z2 - 23)(Z3 — 24) ca (Zor — 22k+1) (8) 
-exp(k(—Z, = 20) or 2 ooo a Bp = k(z +2Z9+23—...+ Z2n+1)) 
d*z,d* zo ane dont 
(ae ea 
a(k,k,t) = a(k, ke +k )t, 
Proof. 


1) Let us compute gd-10(|q|*) for q(z,Z,t) given by (6 ): 


oo) 
gO *A(|q|?) = -8 > 
ill 
a(ky, ki, t)a(ke, ko, t) O58 a(kent1s kona t) 
(ki — ko) (ko — k3)(k3 — ka)... (ken — kon41) 
-exp(Z(—ky + ko — eee Ht Kon - koa) — z(—ky +kg—...+ kon — kon+1)) 


S (Komi+1 = kom, +2)(kom,+2mo+2 = kom, +2m+3) 
my ,,m2>0 
my+tmg+lcn 
kom, +2 = kom, +3 Aecae— kon41 
kom,+2 = kom, +3 ase kon41 
Ee a Ge 
qent+l 


In the sum over m1, mz let us sum over mo first, then the second multiplier 
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and the denumerator cancels: 


Ss (kom,+1 ca kom, 4.2) (Kom, +2m2+2 — oie 3) 


my,mg>0 
mytmoticn 
Komi+2 — Kemi+3 + +++ 7 Kenta 
kom,+2 — kom,+3 +--+. — Kent 
n—-—1 
= De (Koms+1 — kom, +2) (Kam,+2 — Kam,+3Kom. 44 — komits +--+ + kan — kon+1) 
m,20 
Kami+2 = Kami43 + ++ kon41) 
Koma — Kamat + --- — Kong 
n—-1 
= SS (Kom, +1 — Kom,+2)(kem,+2 — Kom.43 +---— kont1) 
m ,=0 
aT nr Tr 
5 (ts — ko +kg —...+ kong)? — So kop? + SS tye 
p=1 =0 
Let us substitute q(z, Z,t) as a functional of {a(k, k,t)} in the equation 
. i WE x, k ky 
(iO; +tkhy+ kyj)a(ki, ky,t) exp(—Z}, ky + 21k) = 
oO nr Tr 
ae) (1 + 3a + Ban) — Slbae? + a) 
n=1 p=0 p=1 
a(k, k, t)a(ka, ke, i) even a(ken+11 kan+1) t) 
(ky —_— ke) (ke = k3) (kg = ka) ks (kan _ kon+1) 
exp(Z(—ki + ko —... + kon — kon41) — 2(—ky + ko —.-. + kon — ken41)) 


d?ky...d?kan41 


qentl 0 


if a(k,k,t) = a(k, Be aes : 7 
2) Substitution of a(k,k,0) = a(k, k), where a(k,k) is given by (8 ), into 
(7) gives q(z, 2,0) = 4(2, 2). 
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Elliptic solutions of the Yang-Baxter equation 
and modular hypergeometric functions 


Igor B. Frenkel and Vladimir G. Turaev 


Introduction 


Various results in algebra, analysis, and geometry can be generalized by 
replacing the ordinary numbers (integer, real or complex) by their trigono- 
metric analogues. For x € C, the trigonometric number [z];, € C is defined 
by 


_ sin(thz) 
(0.a) eS maar 


where h € C\Z is a fixed parameter. It is clear that lim leer us, 
[x], may be viewed as a one-parameter deformation of x. The trigonometric 


numbers are not additive: generally speaking [z+y]h # [z]nt+l[yln- However, 
they satisfy a kind of additivity of “second order forany 2, /,2°< @; 


(0.b) [a+ z]n (2 — 2), = [c+ yla le - yjn + ly + 2]n ly — 2]h- 


Many identities between ordinary numbers can be proved using only the 
additivity of second order and therefore allow a trigonometric deformation. 

It is well known that trigonometric functions admit one parameter gen- 
eralizations - elliptic functions. This suggests the notion of an elliptic num- 
ber [z]n,, € C defined for any z,h,7 € C with h ¢ Z+Zr, Im(r) > 0 
by 


611 (hz,T) 


(0.c) Clabes 


where 6}; is an odd Jacobi elliptic theta function (cf. Section 1.1). It follows 
from definitions that . pli [c]n,r = [z]n. Therefore, we can view the el- 
T)—++00 


liptic numbers as one-parameter deformations of the trigonometric numbers 
and as two-parameter deformations of the ordinary numbers. The elliptic 
numbers do not satisfy formula (0.b) but satisfy the following additivity of 
the “fourth order” 


[2+2][2—2]ly-+w]ly—w] = [2+u)le—y] [z+u][z—w]}+[2+w][2—w][yt2][y-2)- 
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This is a version of Riemann’s theta identity. For trigonometric numbers, 
it follows from (0.b). 

The identities that can be proven using only the additivity of the fourth 
order are much more exceptional than those that can be deduced from (0.b). 
An important class of such remarkable identities appear in statistical me- 
chanics, specifically, in the theory of elliptic solutions of the Yang-Baxter 
equation with spectral parameter. The elliptic solutions were first con- 
structed by R. Baxter [Ba], who deduced the Yang-Baxter relation from 
Riemann’s theta identity. Baxter’s solutions were substantially generalized 
in [DJMO]. We call the solutions of [Ba] and [DJMO] elliptic 6j-symbols. 
They can be viewed as far reaching generalizations of the classical 67- 
symbols introduced by G. Racah and E. Wigner in the early 1940’s (see [BL] 
for a survey). Besides the 6 integral parameters (as in the Racah-Wigner 
theory), the elliptic 67-symbols depend also on three complex parameters: 
h,t, and the spectral parameter z € C. The elliptic 6j-symbols are the 
main object of our study. 

Converging h,7,x to appropriate limits we can degenerate the ellip- 
tic 6j-symbols into less powerful but better known functions. In the limit 
Im(T) — +00, the elliptic 6j7-symbols degenerate into the trigonometric 
6j-symbols studied in our previous paper [FT]. In the limit h — 0, the 
trigonometric 67-symbols yield so-called rational 6j-symbols. The elliptic, 
trigonometric, and rational 6j-symbols are solutions of the Yang-Baxter 
equation with spectral parameter. Finally, we can eliminate the spectral pa- 
rameter xz by converging it to infinity and obtain in this way solutions of the 
Yang-Baxter equation without spectral parameter. In the limit x — +00, 
the rational and trigonometric 6j-symbols degenerate into, respectively, 
the classical Racah-Wigner 6j-symbols and their g-deformations known as 
quantum 6j-symbols (see [KR]). Observe that the spectral parameter of the 
elliptic 6j-symbols lies on the elliptic curve determined by t. Therefore, 
there is no direct way to consider its infinite limit and to define elliptic 
67-symbols without spectral parameter. 

We establish in this paper several new results concerning the elliptic 
6j-symbols. First, we prove the tetrahedral symmetry of the elliptic 6)- 
symbols analogous to the well known symmetry of the classical and quan- 
tum 6j-symbols and generalizing our results in the trigonometric case [FT]. 
This symmetry yields an example of a deep identity between trigonomet- 
ric numbers that generalize to elliptic numbers. Secondly, we establish a 
relation between the elliptic 6j-symbols and the trigonometric 6j-symbols 
constructed in [FT] using a graphical calculus. 

Thirdly, we describe the transformation properties of the elliptic 67- 
symbols under the natural action of the modular group SL2(Z) in the 
space of parameters. It is interesting to note that the naive deformation 
of quantum 6j-symbols obtained by replacing trigonometric numbers by 
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elliptic numbers in the standard explicit formulas does not give the elliptic 
6j-symbols and breaks all non-trivial identities (and also lacks the modular 
invariance). 

An important aspect of the theory of 6j-symbols is their relationships 
with hypergeometric series. The quantum 6j-symbols can be expressed as 
balanced basic hypergeometric series 4¢3, see [KR]. By [GR], they can also 
be rewritten as very-well-poised series g¢7. It was established in [FT] that 
the trigonometric 6j-symbols (with spectral parameter) are balanced very- 
well-poised series 19¢9. Our study of elliptic 6j-symbols naturally leads to 
a generalization of basic hypergeometric functions obtained via replacing 
trigonometric numbers by the elliptic numbers. We restrict our attention 
to balanced very-well-poised elliptic series. This is exactly the class of 
“elliptic” hypergeometric functions with nice modular properties! We call 
these elliptic series modular hypergeometric functions. 

The hypergeometric series 199 yields the simplest balanced very-well- 
poised basic hypergeometric series that are not reduced to a monomial. The 
corresponding modular hypergeometric functions turn out to be a disguised 
form of the elliptic 6j-symbols. The tetrahedral symmetry of the elliptic 
6j-symbols implies an analogue of the Bailey transform for these modu- 
lar hypergeometric functions. We show also that the Jackson summation 
formula for balanced very-well-poised series g@7 extends to modular hyper- 
geometric functions. It is natural to conjecture that other known identities 
involving balanced very-well-poised basic hypergeometric series admit gen- 
eralizations to the modular hypergeometric functions. 

Although 6j-symbols with spectral parameters were first constructed 
in the context of statistical mechanics, they are deeply related to the rep- 
resentation theory. The rational 6j-symbols arise in the finite-dimensional 
representation theory of the Yangian Y(slz). The trigonometric 6 7-symbols 
arise in the finite-dimensional representation theory of the quantum group 
U,(sl2). It is 

interesting to find a representation theoretic description of the elliptic 
6j-symbols. Since the fundamental work of E. Sklyanin, there was a consid- 
erable progress in this matter, however, the existing interpretations require 
ad hoc ingredients. In this paper, we do not discuss this problem, although 
it was one of our motivations. 

This paper consists of 5 sections. In Section 1, we recall the definition 
of the elliptic 6j-symbols following [Ba] and [DJKMO]. In Section 2, we 
give an explicit formula for the elliptic 6j-symbols revealing their tetra- 
hedral symmetry. In Section 3, we relate the elliptic 6j-symbols and the 
trigonometric 6j-symbols constructed in [FT] using the graphical calculus. 
In Section 4, we describe the behavior of the elliptic 67-symbols under the 
modular transformations. In Section 5, we introduce modular hypergeo- 
metric functions and prove the analogues of the Jackson summation and 
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the Bailey transform. 


1. Elliptic 6j-symbols 


1.1. Theta function. Recall the definition of the theta function 0); (see 
[Mu, Section 5]). This is an analytic function in two variables s € C,r € C 
with Im(r) > 0 defined by 

does 1 1 
OMe e) = Se exp(7V —1(n + m) T+ 2nV—l(n+ ale + 5): 


nez 


For a fixed 7, the zeros of 9; form the lattice Z + Zr. 
Recall the well known transformation properties of 6): 


(1.1.a) O14(x ae LF) = Sis Gb). 


(1.1.b) 611(2 + 7,7) = —exp(—aV—17 — 2nV—12) 041 (2,7), 


(ales) 61: (x,7 + 1) = exp(aV—1/4) 011 (z,7), 
(1.1.4) ne emi ermrenpeet exp(@X) 611(zx,T) 


-,-- 
T oF 


where the square root of —\/—17 is chosen so that its real part is positive. 
The function 6,; admits an infinite product expansion given by 


011(z, 7) = —2exp(mV—17/4) sin(rz) II (1 — p™)(1 — 2p™ cos(x) + p?”) 
m>1 


where p = exp(27./—17) (see [Mu, Section 14]). It follows from this formula 
that 


(1.1.e) : jn exp(—mV—17/4) 61:(x,7) = —2sin(rz). 
We shall systematically use the following notation. Fix complex num- 
bers h,r with h ¢ Z+ Zr and Im(r) > 0. For a complex number z, set 


611 (ha, T) 


(1.1.£) l= [he = Fa 


EC. 
Observe that [—a] = —[s] and [1] = 1. 

Unless explicitly stated to the contrary, we shall assume that h ¢Q+ 
Qr. This assumption ensures that [n] # 0 for any integer n 7 0: 
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1.2. Baxter’s elliptic 6j-symbols. R. Baxter [Ba] introduced elliptic 
6j-symbols as solutions to the so-called Yang-Baxter equation. Baxter’s 
6j-symbol is a complex-valued function of 2,h,r depending on 4 integer 
parameters 7, j7,k,/ and denoted here by 


(1.2.a) Ik } {| (eh). 


We shall usually fix h,7 and omit them from the notation for the 67-symbol. 
The number (1.2.a) is defined as follows. It is equal to 0 unless 7, 7,k,1 > 
0 and 
li — | = [9 —h] = |e - 1] = [L- a] = 1. 


Under these conditions, we have i =14£1,k =1+1, andj =lorj=/+2. 


Set 
ise il tee? Il 
Ee | : (x) = [x + 1], 


ee l i|@= (i+1+1][z} 


(ee (+1)? 
ee l i] @ = Saas 
i+t1 1 1 (i+) 


Since h ¢ Q+Qr, all the denominators on the right hand side are non-zero. 
Note that the 6j-symbol (1.2.a) is an elliptic monomial, i.e., a product 


+[[(rl [[fe +1” 


Pees seZ 


with integer (possibly negative) a,,b, equal to zero except for a finite set 
of r,s. (The factor [1] may also occur but we can ignore it since [=a 
The degree >>, bs of the monomials defining Baxter’s 6j-symbols is equal 
to 1. 


£ k 
tae ie 
ie) ape : 
a i 
Figure 1.1 


It will be convenient to represent the 67-symbol (1.2.a) graphically by a 
square with the parameter z sitting inside and the numbers i, j, k, | attached 
to the vertices in the anticlockwise direction beginning in the bottom left 


corner, see Figure 1.1. 
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It is easy to check that under the opposite choice of direction we get 
an equal 67-symbol, see Figure 1.2. 


e k j k 


Figure 1.2 


Our notation differs from the one in [DJKMO] where the square brack- 
ets are defined by [x] = 6;(hz,7) and the 6j-symbol (1.2.a) is denoted 
Wii (2,9, k,1|2). The complex parameter € used in [DJKMO] is set here to 
be 1. For this value of €, the number W); (2, j,k, 1 |x) is defined in [DJKMO] 
by the same formulas as above, for any i,7,k,1 € Z with 1 # —1. We shall 
consider only non-negative 2, 7, k, 1; this will allow us to consider tetrahedral 
symmetries of 6j-symbols involving, in particular, permutations of | with 
1,j,k. Our notation is meant to emphasize this symmetry. 


1.2.1. Theorem. ([{Ba], [DJKMO]) (i) For any i,j,k,l,e,f > 0 and 
t,y€C, 


ay Dl) Fei? ijetw[2 7 te- 


-D[f i tlmli 2 tle+n[f * ie. 


(ii) For any i,j,k,l >0 andxe€C, the sum 
teege L gq 9 1 
(1.2.c) =I i|@+n |g ; | @ 


does not depend on the choice of e such that |e — k| = |e —1| = 1. 


Formula (1.2.b) is called the Yang-Baxter equation. Its graphical in- 
terpretation is given in Figure 1.3 where in both parts we multiply the 
6j-symbols associated with the three (distorted) squares and sum over the 
integer variable g = 0,1, 2, ... assigned to the vertex drawn in bold. Formula 
(1.2.c) plays a key role in the fusion procedure, see Section 1.3. Note that 
the sums in (1.2.b), (1.2.c) are finite, since their terms are equal to 0 except 
for a finite number of values of g. 
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Figure 1.3 


Proof of Theorem. Formula (1.2.b) follows from the formula 


S$) Wilt, 5,9, f |) Wiilf,g,l,e|z +y) Wiig, J, k,1|y) = 
g 


= S>Wilf,i9,e1y) Wilt, J, k,g\x a y) Wig, kde) 2) 
9 

established in [Ba]. In the last formula g may take negative values but 
the corresponding terms are equal to 0. Indeed, the g-th term on the 
left hand side is 0 for g < —2 since in this case |j — g\ = 1 the term 
corresponding to g = —1 might be non-zero only ifi =e=k=1 andj = 
| = f =0. However, in this case it also equals 0 because W11(i, 7,9, f |x) = 
W41(1,0,-1,0|y) = 0. A similar argument applies to the right hand side. 

For k > 1, the claim (ii) of the theorem follows from [DJKMO, Lemma 
2.1.1 (i)]. For k = 0, there is nothing to prove since there is only one choice 
e=1 fore. 


1.3. General elliptic 6j-symbols (see [DJKMO]). The general elliptic 
6j-symbols are complex-valued functions of x,h,T 


ie fet Jann 


depending on six non-negative integer parameters 1, j,k, 1,m,n with m,n > 
1. We shall usually fix h,r and omit them from the notation for the 6)- 


symbol. 
The number (1.3.a) is defined as follows. It is equal to 0 unless 


i,j, k,l,m,n > 0 and 


i—-j,k—le {—n,—n+2,-n+4,..,.n —2,n}, 
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1—l,j—ke {-—m,-—m+2,-m+4,...,m—2,m}. 


If these conditions are satisfied, then one can find a pair of sequences of non- 
Negative integers Goin, = !,01m,--.@nm =| 9 and o,,, = &, 
Qn mn=l,--;@n,o = 7 such that the neighboring terms difier by 1) Pix 
such sequences and consider their extension to an integer (n+ 1) x (m+1)- 
Iman icedu sles), . Withise—) 0, la wate — 10 erectich thatecro ye a7 
and | ast — @s41,t| =|Os,t — %s,t41 | = 1 for any s,t. Set 


MAL 2 Fh 


ign = 


? 


and 


a - il 
st a +1,t ‘ (c+ Rea) 


n—-Im-1 
[eae Os+1t+1 


(1.3.b) i : a («)=S-T] I] 


A s=0 t=0 


where the sum is taken over all matrices A as above (with values i, 7,k,] 
in the corners and fixed last row and column). A graphical expression for 
this sum is given in Figure 1.4 where we multiply the 6j-symbols associated 
with the mn unit squares and sum over the integer variables assigned to 
the vertices drawn in bold. The 6j-symbol (1.3.a) is graphically presented 
by a square with the parameter z inside, the numbers i, 7,k,1 attached to 
the vertices and the numbers m,n attached to the edges. 


es 
s 


3 
&K 


Figure 1.4 


Observe that there is only a finite number of matrices A contributing 
non-zero to the right hand side of (1.3.b). Repeated use of Theorem 1.2.1 
(ii) shows that the right hand side of (1.3.b) does not depend on the choice of 
the sequence a9 m = 1, @ijm,--+5 Qn,m = k. Similarly, a version of Theorem 
1.2.1 (ii) obtained by transposition as in Figure 1.2 shows that the right 
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Figure 1.5 


hand side of (1.3.b) does not depend on the choice of the sequence n,m = 
k, Qn,m—1) ++) n,o = j- Therefore the 6j-symbol (1.3.a) is well defined. 

We shall say that a triple of non-negative integers 2,7, k is admissible 
if |i —j| < k < [¢ +9] andi+ j +k is even. A 6-tuple of non-negative 
integers (2, j,k, 1,m,m) is said to be admissible if the triples (7, 7,7), (k, 1,7), 
(i,1,m),(j,k,m) are admissible. By definition, the 67-symbol (1.3.a) may 
be non-zero only when the tuple (i,j,k, 1,m, 7) is admissible. 

The next lemma follows directly from definitions. 


1.3.1. Lemma. Let (i,j,k,1,m,n) be an admissible tuple of integers with 
m,n>1. Ifn=nitng with positive n1,n2 € Z then for any non-negative 
p € Z such that the triples (1,71, p), (p,n2,k) are admissible and for any 
reEC, 


ase [i 7 2] @=|, gm) er) [fF mle, 


Ifm=m,+ mg with positive m,,m2 € Z then for any non-negative p € Z 
such that the triples (j,™1,p), (Pp, me, k) are admissible and for any z € Cc 
(1.3.d) 


Ef aje-r[ i alee [tt ale? 


A graphical expression corresponding to formula (1.3.c) is given in Fig- 


regio. 
The symmetry in Figure 1.2 generalizes to the symmetry 


(1.3.6) q d 2 @=|; ; z ): 


Equality (1.2.b) generalizes to the following Yang-Baxter relation easily 
proven by induction. 
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1.3.2. Theorem ({DJKMO)). 
Ge yi BO 2 kel ohm 


“rl FM: 2 pleralt & Ale. 


For completeness, we mention another important identity, the so-called 
orthonormality relation. We need the following notation: for zc € C and 
non-negative integers m,n, set 


rn 


(1.3.g) aie) = 


= 


lm-]1 

m+n 

—— —s—t-—ll. 
NB Ohe 5 8 ] 


This is an elliptic monomial of degree mn. Note that 
Inn (2) = Unjm() = (—1)"" Um n(—2) = (—1)™"Mn,m(—2). 


We have 


=|, j "|e E } a (=e) = (=) e+ Ie oe. 


g m n 
9g 


1.4. Remarks. 1. The assumption h ¢ Q+Qr ensures that all 67-symbols 
(1.3.a) are well defined. For each particular 6j-symbol (1.3.a) to be well 
defined, it suffices to assume that Nh ¢ Z+ Zr for N = 1,2, wyttm+tn. 

2. In [DJKMO] the 6j-symbol (1.3.a) is denoted by Wi, (i,j,k, l| a+ 
(m —n)/2). 


2. Computation of the elliptic 6j7-symbols 


In this section we shall give an explicit formula for the elliptic 67- 
symbols (Theorem 2.2). This formula reveals their tetrahedral symmetry. 
We first introduce the necessary notation. 


2.1. Notation. For an admissible triple of non-negative integers 1, 7, k, 
set 


[Se Ne en 
fl! (a)! (a 


Here for a positive integer a we have [a]! = [1][2]...[a] and for a = 0 we 
have [a]! = 1. Note that @(i, j,k) is an elliptic monomial of degree 0. (The 


(2.1.a) O(i, j,k) = (—1)0+9+4)/2 
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corresponding trigonometric monomial naturally appears as the bracket 
function of the so-called 6-net, cf. Section 3, [KL], [FT].) 


2.2. Theorem. Let (i,j,k,l,m,n) be an admissible 6-tuple of integers. 
Set 


atk Ge peek om 
aime game 
and a 
g=at+h+y= 
Forx€C, set 
(2.2a) ii slo- 
ene ee rele 
il ([9]}2 OGG, 2, 1) O(9,2,0)9(9,1,8) 3 IT te a o7 7 + slo + 9S +145) 


where g runs over non-negative integers such that the triples (Gyn, 7) (g52, 2), 
and (g,l,) are admissible. Then 


(2.2.b) 
ij _ sasiowes ees IE elles 
R l rf@=Cn Ce]! (ro)! [=A [A ! 
1 oo AB 41 ae ’ 
<Onaate+d) [[ et? TT ett |; 7 2) ©. 
r=—itt r=t5241 


The proof of this theorem is given in Section 2.4. It is based on the 
following lemma whose proof is postponed to Section 3. 


2.3. Lemma. 


(2.3.a) B aac | (2) = 
cag (i+ 1) (EEE EY! (=? 


= (0 Tea Ea TOG Lm) 


a t 


imat 
x Tnn- ule = 5) TI fer14 229 I [z ee —r], 


r=1 


(2.3.b) ee J "|@= 


182 I. B. Frenkel and V. G. Turaev 


iptem [SES 2) (EY? Ff + ml! [i+ O(n, 5 + m) 


==) ea Ses Te 
[eee] |! (el)? OG, 7, 2) OG, tm) 
oe timo! 
1 j+m—i (ae § 32000) 
xImn-(@+5) J] fae OU eS 
a ill 


2.4. Proof of Theorem 2.2. Set in formula (1.3.d) p = y,m, = a, mg = f. 
We have 


ii sjo-x[f 2 tleromls 7 g]e-o 


Note that 7 = y+a and k = y+ so that the 6j-symbols on the right 
hand side are the ones computed in Lemma 2.3. We obtain 


pee esl a eS ee 
tall = CW (=a ou ne) 
x Tan ae + <4) I tit 2325 I Ip t24+ 249 _ nx 
(12 EE EI? (a! ig + 1O (1, rs &) 
(A! Gl)? HOG. 7.2)0@.48) 
a Ue eee al 


To simplify this expression we observe that 


= 


B+ a 1 1 
5 ) = Tet gn—1(2+ m) = Wmjn-1(2+ 5) 


2 


i 
Ilgn-1(2+ ) Ign—1(@+ 


and that the product of the other expressions involving x is equal to 


ar ee. o i 
= = (on — 
etry TT ete Te - 7 4 oie t 22 414 gh, 
r=— itt ra5f41 s=0 2 


We aslo replace O(k,1,n) using its definition. This gives formula (2.2.b). 
(To compute the sign one should use that g = 8 — (mod 2).) 


2.5. Symmetries of the elliptic 6j-symbols. Theorem 2.2 shows that 
the expression (2.2.a) yields the the elliptic 6j-symbol (1.3.a), at least up to 
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multiplication by a monomial. It is easy to see that the 6j-symbol (2.2.a) 
is invariant under 6 permutations generated by the following two 


(2.5.a) oe eat elo- el 


As in [FT, Section 8.10], it is natural to associate with the 6j-symbol (2.2.a) 
the geometric picture consisting of a triangle whose center is connected to 
the vertices and endowed with z € C, the six edges of the resulting graph 
are labelled by i, j,k,1,m,n as in Figure 2.1. The symmetries in (2.5.a) are 
induced by the automorphisms of this graph preserving its central vertex. 

Together with (1.3.e) the symmetries (2.5.a) generate the full symmet- 
ric group on 4 elements. Thus, the elliptic 6j7-symbols considered up to 
multiplication by monomials have the full tetrahedral symmetry. 

It is easy to deduce from definitions and the identity [—2] = —[z] that 
the 6j-symbol (2.2.a) is invariant also under the transformation x ++ —1—z. 


i 
Figure 2.1 


3. Comparison with the trigonometric 67-symbols 


3.1. The trigonometric 6j-symbols. In [FT, Section 6.3] we defined 
the trigonometric 6j-symbol 


(3.1.2) ee m} wee 


where (i,j,k, 1, m,n) is an admissible tuple of integers and h € C\Z,z € C. 
The definition of the trigonometric 6j-symbol given in [FT] relies on the 
ideas of the theory of skein classes of knots. In Section 3 we assume that the 
reader is acquainted with this definition. For completeness, we reproduce in 
Section 3.4 the explicit formulas for the trigonometric 6j-symbol obtained 
in [FT]. Note that the 6j-symbol (3.1.a) is computed via the function 


sin(whz) _ exp(1/—Ilhz) — exp(—1/—Ihz) 


Ca) le}a = sin(th) exp(1/—1h) — exp(—t/—Ih) 
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In the notation of [FT] we have gq = exp(7/—1h), g? = exp(7/—1hz), and 
[z]n = (q7 — q~*)/(q — 971). Here we shall not use this notation. 
It follows from (1.1.e) and (1.1.f) that 


; O11 (hz, T) sin(7hz) 
: -= ] SS Se = : 
(3.1.c) eae [z]n, iA ee 61, (h,T) sin(7h) ialp 


3.2. Theorem. Let (i,7,k,l,m,n) be an admissible tuple of integers and 
xeEC. Then 


' se ~ 
(3.2.a) eee re f im (ei) 


= (— mn-+t 


[b+ tn [eat da! Sa! . j ay es 
[Jn! (=)! (= ),! GT) n) A(, L, m) k lo om h 


where (i,j,k) is defined by the formula (2.1.a) with the elliptic square 
brackets [...] replaced everywhere with the trigonometric brackets [...]n. 


Here for a positive integer a we have [a],! = [1],[2]n...[a], and for a = 0 
we have [a];,! = 1. 

Note that the 6j-symbol on the left hand side of (3.2.a) is well defined 
if h ¢ Q and Im(r) is big enough, say, Im(r) > (¢ + m+n) |Im(h)|, ef. 
Remark 1.4. 


Proof of Theorem. It follows from formula (3.1.c) that in the case 
1 —letineslinart 


: fp 
wee R l af By 
exists and is given by the same formulas as in Section 1.2 with the only 
difference that the brackets [...] should be replaced everywhere with [...]p. 
It follows from this fact and the definition of the elliptic 6j-symbols that 
the limit on the left hand side of (3.2.a) exists for all m,n. (This limit does 
not depend on Re(r).) 

Let us compute the right hand side of (3.2.a) for m =n = 1. We can 
assume that |i — j| = |j — k| = |k —1| = |1 — i] = 1. By definition of the 
trigonometric 6j-symbol, we have the equalities in Figure 3.1. The right 
hand side equals 


7 Ok LA) dag 4 (k, ce 1) adGgol) 


) ate 
et Gaye 1g enyge a, 
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It is easy to compute that for 7 =i+1, 

(3.2.b) O(é, 7,1) = O(j, 4,1) = (—1)** "fe + Qa. 

Now, a direct computation yields that 


pee EEO), 
ee l ‘| (x) = (-1)'***)[1 + 2+ Male — Ma, 
h 


ee ; i : \ (x) = (1)? + 2jalz)a, 


l+1 1 1 oS [i+ 3/2+ 1/2] 
{i i} @= CU piesa, 


Substituting these expressions and (3.2.b) in the right hand side of (3.2.a) 
and observing that [1], = 1 we obtain precisely the formulas of Section 
1.2 for the Baxter 6j-symbol with the brackets [...] replaced everywhere by 
[...J,- This proves (3.2.a) in the case m=n= 1. 


(0+1tc}p. 


Figure 3.1 


The remaining part of the proof goes by induction. Denote the right 
hand side of (3.2.a) by 


(3.2.c) ir : le) 
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It suffices to establish for this function inductive formulas analogous to 
(1.3.c) and (1.3.d). Moreover, by [FT, formula (6.3.g)] the right hand side 
of (3.2.a) is invariant under the involution m > n,n mle j,j 1. 
Therefore it is enough to establish the following analogue of (1.3.c) 
Gd) ' 

oH i 7g nm n2 Gj no ny 
{i I OO I mf er fe p | Ce 
where (i, 7,k,1,m,n) is an admissible tuple of integers with m,n > 1, n= 
n, + ng with positive n,ng € Z, p is a non-negative integer such that the 
triples (1,71, p),(p,n2,k) are admissible, and z € C. Such a formula was 
essentially obtained in {[FT, Section 8]. Indeed, by [FT, formula (8.3.b)], 


a9 mtu (tT TALE a},eo- 


Mm Ng n tg ny Se a ie: ah 
e 2 ae l oaAG at e ah et® 
9(9, p,m) (9, 21, t) O(g, n2, J) 


are the trigonometric 67-symbols without spectral parameter corresponding 
to q = exp(x/—Ih), cf. [FT, Section 6.2]. 

The role of the numerical factor appearing in front of the trigonometric 
6j-symbol in (3.2.a) is to transform formula (3.2.e) into (3.2.d). Since 
n =n, +n, we have 


ee nz nt = O(k, L n) (He ),! [Ate=n2),,! 
a 2 [pln! ($= Ih! 
(cf. [FT, formula 6.2.c]). Similarly, 
1 no 7 _ 2G pee Se 
Fe alle lel 2a 


Substituting these expressions in (3.2.e) we obtain a formula equivalent to 
(3.2.d). 


3.3.Proof of Lemma 2.3. We first briefly discuss relationships between 
elliptic and trigonometric monomials (cf. Section 1.2). A trigonometric 
monomial is a product 


+ [I (layer [][ (2 + sla) 


r=2,3,... seZ 
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with integer a,,b, having non-zero values only for a finite set of r,s. (The 
factor [1], may also occur but we can ignore it since [1], = 1.) Clearly, 
there are no non-trivial identities between trigonometric monomials: two 
trigonometric monomials are equal as functions of x and q if and only if they 
correspond to the same sequences {a,},, {bs }s. The formula [...]n,r +> [---]n 
establishes a bijective correspondence between elliptic and trigonometric 
monomials. For instance, the elliptic monomial O(¢,7,k) corresponds to 
the trigonometric monomial 6(i, j, k). 
By [DJKMO, formulas 2.1.14, 2.1.22al, 


(3.3.a) i ale 


is an elliptic monomial. (Formula (2.1.22a) in [DJKMO] holds for m < n, 
the case n < m is similar.) By (3.1.c), the corresponding trigonometric 
monomial represents 


: i kim n 
(3.3.b) al i i a (x). 


We can compute the latter monomial using Theorem 3.2. By |[FT, formula 
(8.8.a)], 


i k+mi n _ 7 tmsi¢mn Jt k+m n 
(io ec lala inne 


m 
itmet k F Lemos ae 
1 +m— ™m 
XTmn—1(t+5) Ul ae ale I ee 


where Tm,n is defined by formula (1.3.g) with the elliptic square brackets 
[...] replaced with the trigonometric brackets [...]n- Substituting here 


ie l it ion I i (On! 22" I,! 


and substituting the resulting expression in (3.2.a) we obtain a trigonomet- 
ric monomial representing (3.3.b). By the argument above, the correspond- 
ing elliptic monomial represents (3.3.a). A direct computation shows that 
this monomial is exactly the one on the right hand side of (233-4). Ehe 
proof of (2.3.b) is similar using [FT, formula (8.7.a)]. 


3.4. An elliptic deformation of the trigonometric identities. The- 
orem 3.2 clarifies the relationships between the elliptic and trigonometric 
6j-symbols and allows us to compare the results of Section 1 with similar 
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results of [FT]. It is easy to check that the Yang-Baxter identity for the 
elliptic 67-symbols (Theorem 1.3.2) in the limit Im(r) — +00 becomes the 
Yang-Baxter identity for the trigonometric 6j-symbols |FT, Theorem 6.5]. 
The orthonormality relation formulated in Section 1 degenerates to the or- 
thonormality relation for the trigonometric 6j-symbols [FT, Theorem 6.4]. 
The symmetries of the elliptic 6j-symbols degenerate to the symmetries of 
the trigonometric 6j-symbols established in [FT]. We expect that any al- 
gebraic identity for the trigonometric 67-symbols with spectral parameter 
admits an elliptic deformation. On the other hand, the identities for quan- 
tum 6j-symbols (which are limits of the trigonometric 6j7-symbols when the 
spectral parameter converges to infinity) break down when one replaces the 
trigonometric numbers with elliptic ones. This phenomena is deeply related 
to the modular properties of the elliptic 6j-symbols studied in Sections 4 
and 5. 

The trigonometric 6j-symbols and the identities between them have a 
geometric interpretation given in [FT] in terms of nets. This interpreta- 
tion does not admit any straightforward elliptic generalization. It is an 
outstanding problem to find a geometric interpretation of the elliptic 6j- 
symbols and the identities between them. 

The trigonometric 6j-symbol (3.1.a) was explicitly computed in [FT, 
Theorem 8.10]: in the notation of Theorem 2.2, we have 


maint 1 E : 2 a aS ’ 
(=1) °F tm n—1(t—5) [I etre? 7] fete? + j | (a) 
r=i;4 ra HE] A 
where 
Mal ail Foto 
ee Et a als 
s=0 t=0 
and 
. & / 
i n 
(3.4.a) { ee = (x) = 
h 


= t @ g ie ney Gg 
Tlo+th {5 I eae : ee ; aes 


x (6(9, 8,1) (9,7, 7) (9, 0,4)" TT fe + 4 - 5 — ta [e+ ne rele 
s=0 


Note that three 6j-symbols without spectral parameter, appearing on the 
right hand side, are monomials computed as at the end of Section 3.2. 


Yang-Bazxter equation and modular hypergeometric functions 189 


The trigonometric 67-symbol (3.4.a) has the same symmetries as the 
elliptic 6j-symbol (2.2.a) (cf. formulas (2.5.a) and [FT, (7.4.a)]). These 
6j-symbols are related by the following formula that can be derived from 
the definitions and (3.1.c): 


. 6 / 
mers (oma oD a 
ee. li l i Ds 


A(i,1,m) (k,l, n) O(4, 5, 2) : j nh eo, 


[em |), (a! k lo m 


We would like to use this opportunity to correct a few errors in the 
figures in [FR]. On the right hand side of Figure 4.2, the number 2;,; as- 
signed to the (7, j)-crossing should stand not on the right of the crossing but 
exactly below it (as on the left hand side of this figure), here 7 = 1,...,m 
and j = 1,...,n. In Figures 6.8 and 6.9 the symbols x and —z should stand 
below the corresponding crossings. The same remark concerns the positions 
of z on the left hand side of Figure 6.10, r2 in Figures 8.2, 8.3, 8.4, 8.5, and 
z1 in Figures 8.4, 8.6. In Figure 8.1 the symbols m,n should be replaced 
with 1. In Figure 8.8 the symbol Y, should be replaced with Y,. 


4. Transformation properties of the elliptic 67-symbols 


4.1. Basic transformations. Denote by TZ the set of triples (z,h,T) € ce 
such that h € Q+Qr and Im(r) > 0. Fix three complex numbers L, M,N. 
We define a right action of the group SL2(Z) on the functions f :Z — C 


by the formula 
a b 


Lz? + Mzh+ Nh? Z h ar+b 
ct+d er+d’cr+d' cr+d 


exp (-" —le ). 


Analogous actions of SL2(Z) on functions of one variable g(7) and functions 
of two variables g(z,7) given by 


(o[¢ a))@=9G50 


and 


ab _ Pa alle z att+b 
GE DY tar) =e (-* a2) Se a) 


appear in the theory of modular forms and Jacobi forms (see [Mul], [EZ}). 
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Fix ¢ € C\{0}. We consider four actions of the infinite cyclic group 
on the functions f : J — C. These actions are defined by the following 
formulas, where a € Z: 


(fa)(z,h,7) =e~* f(z+a,h,7), 
(fa)(z,h,7) = e7*exp(mV/—1a(Lar + 2Lz + Mh)) f(z + a7, h,7), 
(fa)(z,h,7) = f(z,h+a,7), 
(fa)(z,h,7) = exp(t/—la(Nat + 2Nh + Mz)) f(z,h + a7,7). 


Observe that a function f : J — C is invariant under the action of SL2(Z) 
and the actions of Z if and only if it satisfies the following six identities: 


(4.1.a) fizZ@ i hae f(z,h.r) 
(4.1.b) i ol a alga) 1 CZ Ie ca 
(4.1.c) (2, Weta) — (trie 


(4.1.d) f(z+7,h,7) = eexp(—mV—1(Lr + 2Lz + Mh)) f(z,h, 15))p 


(4.1.e) f(z,h+7,7) = exp(—rV—1(Nr + 2NA+ Ma) fi zatia), 


h Lz? + Mzh+ Nh? 
a mfp) Flesh, r). 


(41) — #(2,7,-2) = exp( 


4.2. Theorem. Let (i,j,k,1,m,n) be an admissible tuple of integers. Then 
the function 


(4.2.a) flehr) =|} } i 


is wnvariant under the action of SL2(Z) and the first three actions of Z 
corresponding to 
L=mn, 


(4.2.b) FV ey te ti ine ail in) pasa tl)” 
2 ’ 
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2 2 Hee oo 2 
(4.2.0 mn(m — n) _mG+) + (k+1)?—-(¢+1)?-(1+1) i 
2 2 2 
n(k+1)? +(1+1)? — (i+ 1)? — (+1)? 
oa a 

2 2 
and ¢ = (-1)™". If m+.n is even then the function (4.2.a) is invariant 
also under the fourth action of Z. 


Note that M € Z. Indeed, the admissibility of the triples (i, 7,7), (k, 1, ) 
implies that 7 + 1 = i+k(mod2). In the case m = n the formula for N 
simplifies to 
_ oy +1? = (+ D" 


2 


Proof of Theorem. It is convenient to write down an explicit formula 
for the function f = fi,;,k,1,m,n defined by (4.2.a). In the case m =n = 1, 


we have f) 
Zap ae tf 
fiaait211,1,1,1(2, 2,7) = ae 


Aii((l + 1+ Ly) 641(z,7) 
h, Se eeeeeeeeeeFeFeFfFfsfse 
fix1,l,1£1,1,1,1 (2 i) Air((l ae 1)h, T) 611 (h, T) 


ae, A11(z + (I =F 1)h,7) 
eee alee T) = 631,((1 + 1)h, 7) 


Formula (1.3.b) yields the following formula for f = f:.j,k,lm,n m the case 
mee 2 or nt > 2: 


n—-1lm-1 
(4.2.d) es h, 7) = SZ II I] Fe Ce corre ee Ul cataitans h, T) 
Aas —Ont—0. 
where ae 
eee = =o t ad 1 


and A = {as,z}s,¢ runs over the same set of (n +1) x (m+ 1)-matrices as 
in (1.3.b). 

Now we can check equalities (4.1.a) - (4.1.f). For m =n = 1, formula 
(4.1.a) with e = (—1)™" = —1 directly follows from (1.1.a) and the formulas 
for fi,7,k,1,1,1 given above. The general case follows from (4:2.d). 

For m = n = 1, equality (4.1.b) directly follows from (1.1.a) and the 
formulas for fi,;,4,1,1,1 given above. The general case follows from (4.2.d). 

In the case m = n = 1, equality (4.1.c) follows from (1.1.c). The general 


case follows from (4.2.d). 
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For m = n = 1, formula (4.1.d) is verified straightforwardly case by case 
using (1.1.b). The general case follows from (4.2.d) using the equalities 


n—1lm—-1 
(4.2.e) R.,=0 
s=0 t=0 
and 
(4.2.1) 
eee [2 ee (sebe n—1m-—-1 
s=0 2. 
where 
ds, =ds(A) = (Gorter YE Gea Ue See (as41,t41 + 1)? 
st = 4st Ee ea er ee 


7 = coe y 


Let us prove (4.1.f). First consider the case m = n= 1. A direct case 
be case computation based on (1.1.d) shows that 


(4.2.g) 
42,2,-+) =e (=v=i 


22+ Mzh+ ay 
eee Cao 

T 
where M is given by (4.2.b) with m =n = 1. This proves (4.1.f) for m = 
n = 1. Consider now the general case. We fix sequences of non-negative 
integers G9,m =1,Q1,m,---)Onjm =k and Oy m =k » Onsm—I> 2.0 = 7 such 
that the neighboring terms differ by +1. Both f(2,2,—+) and f(z,h,r) 


27? 
are sums over matrices A = {a,+} extending these sequences. It follows 


from (4.2.g) that the summands corresponding to one and same matrix A 
are proportional with coefficient exp(2¥=! p) where 


a 


n—1lm—-1 5 
SS ((e + Resh)(z+ Rosh +h+ dosh) + Soo aa eee n?) 
t=0 


s=0 
n—1lm-l1 ( oT 2 

=mnz*4+Mzht). So (ce. eRe Se) 42 
s=0 t=0 


where the last equality follows from (4.2.e) and (4.2.f). 
A direct computation shows that 


n—1m—-1 


_ mn(m —n)? 
> Se Cia is a a 
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It is easy to check that all the entries of the matrix {as 2} contribute 0 to 
the sum 


n—1lm-—1 
SY (Resdoy + Geese tO lene a 


s=0 t=0 yy 


except the four corner entries a9,0 = 1,Qn,0 = J, 00,m = !,Qn,m = k which 
contribute respectively 


_(oo0+1)? p  _ (ao0+1)? __ (+1)? mtn 
9 0,0 D a 2D) 2 a 


(Qn,o +1)? (G+1)? m—n 
2 2 ee 
(a0,m +1)? Ro _ (141)? n-m 


2 ay 2 Jae 
- (Qnjm + 12 (Chae cr 2 ve (k+ 1)? m+n 
2 2 - 2 2 
Combining these formulas together we obtain P = mnz? + Mzh + Nh?. 
In particular, P does not depend on the choice of the matrix \Onay-) his 
implies (4.1.f). 
To prove (4.1.e) we note first that for any R € Z, 


00 = 


Jpeg 


(4.2.h) 61:(u+ Rr,T) = (—1)"exp(—1V—1R?7 — QnV/—1Ru) 611(u, 7). 


This follows by induction from (1.1.b). As above, we first prove (4.1.e) in 
the case m =n = 1. Using (4.2.h) we obtain 


fitrpertertar(Z +77) _ CT Cia cs CO 
fia tt2,141,1,1,1 (2) T) 64:(h+ 7,7) O11 (z +h,T) 


= exp(—m/—1(7 + 2z + 2h)) en a 
exp(—7/—1(7 + 2h)) 
Similar computations show that 


h 
F- Pes Od, ? ? 


age ee h SF T, 2) = exp(+2mvV—1(I +4 1)z). 
fizttgti,t1(2, 7) 
In all cases 


2_ (i 2 
opener CanCK es) Dy ee ae ele ee 4 2h) = x/=IMz) 
fo ea2, nh, 7) ; 2 
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where M is given by (4.2.b) with m = n = 1. This proves (4.1.e) for 
m =n=1. We shall need a more general formula concerning fj,5,4,1,1,1- It 
follows from (4.1.d) that for R € Z, 


fig ktaa(Z + Rr, h,7) _ (—1)®exp(—1V—1(R27 + 2Rz + RMh)). 
Jeg ae Oy) 


This implies 


(4.2.1) Fijetia(@+ Rr,h+7,7) = 


peor 


= (-1)"exp(—mV—1(R?r + 2Rz + RM(h+7))) fig,et,1,1(2)2 + 7,7) = 
= (—1)®exp(—1V—1(R?7 + 2Rz + RM(h+17)+ 


(k+ 1)? —(@+1) 
Sp 
2 

Now we prove (4.1.e) assuming that m+n € 2Z. This assumption 

implies that R,, € Z (in the notation introduced above) which will allow 

us to use (4.2.1) with R = R,+. By (4.2.d), both f(z,h+7,7) and f(z,h,7) 

are sums over one and the same set of matrices A = {a+}. It follows 

from (4.2.i) and (4.2.e) that the summands corresponding to a matrix A 
are proportional with coefficient 


(7 + 2h) + Mz)) fi,5,,1,1,1(2, 2,7). 


Hy*eo BVT) = exp XQ) 
with 


n—1lm-—-l1 


nO 0 Re) ee Fh, ee ede) pee) 


s=0 t=0 


i" (Qs41e¢1 +1)? — (a5 +1)? 
2 


The computations made above imply that 


(7 + 2h) + (14+ d,t)(z + Rs th)). 


oe ae : ay Bh ¥ il 2 
a ss (0 Pearce a | (7-+2h) 4 
s=0 


<#=0 2 
n—-1lm-1 
+0 Sl 4dy4)2 = N(r + 2h) + Mz. 
s=0 t=0 


This yields (4.1.e). 
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4.3. Remark. The function f given by (4.2.a) satisfies the identity 
(4.3.a)  f(z,h4+ 27,7) = exp(—7V—-1(4N7 + 4Nh+ 2Mz)) f(z,h,7). 


For even m+n this formula follows from (4.1.e), for odd m+n it is proven 
similarly to (4.1.e). 


4.4. Corollary. it Let (i,7,k,1,m,n) be an admissible tuple of integers. 
The function F :Z — C given by 


a ff WW 


Fehr) =|} | (z,h,7) 


is invariant under the action of SL2(Z) defined by 


(|? i} ) Gir) = 


ae F h ar+b 


(Cuenca) 


exp (-x hen 


This is a reformulation of the first claim of Theorem 4.2. 
5. Modular hypergeometric functions 


5.1. Classical hypergeometric series. The hypergeometric series with 
p numerator parameters a1,...,%p € C and r denominator parameters 
1, ..., By € C is the formal power series in variable z defined by 


fore) (a1)n---(@ Jha a 
RE nye en, Cl er) = yD ReaCae 2”, 


n=0 


where (@)n denotes the shifted factorial: 


ile ni) 
(an = { i 

a(a+1)---(at+n—-1), n=1,2,... 
One assumes that the numbers f1,...,8, are such that the denominator 
factors in the terms of the series are not zero. For |z| < 1, the series 
converges absolutely and yields a hypergeometric function. If one of the 
numerator parameters @1,...,Q@p 1S zero or a negative integer, the series 
terminates and becomes a polynomial in z. 

In 1846/47, E. Heine introduced basic hypergeometric series p¢, es- 

sentially by replacing ordinary numbers by trigonometric numbers in the 
definition of ,F,. Fix a non-zero complex number q. We have 


(5. laa) pr(Q1, ++) Gp; 51, Oe) = 
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=> (4139)n-:- (4p; 9)n ” 
4 (G59)n (b1;q)n-*(brig)n 


where 


iL. C 
(a; Q)n = { (1 = a)(1 = aq’) oe (1 ss aq” *), allan cee 


The expression (a;q), can be easily rewritten as the g-shifted factorial for 
the trigonometric numbers. This allows us to view p¢, as the trigonometric 
analogue of pF. 

Heine and his followers were able to generalize various results about 
hypergeometric series to the basic case, see [S] and [GR]. According to 
the philosophy of three types of numbers outlined in the Introduction and 
supported by the study of 6j-symbols, one can replace ordinary numbers 
(resp. trigonometric numbers) in the definition of hypergeometric series 
(resp. basic hypergeometric series) by their elliptic analogues. In the full 
generality, it is hard to expect that the classical properties of hypergeomet- 
ric functions extend to this new class of elliptic series. Still, as we shall see 
below, the fundamental results concerning the narrower class of balanced 
very-well-poised series do admit elliptic generalizations. 

Recall the definition of very-well-poised basic hypergeometric series. 
From now on, we restrict ourselves to the case p = r +1, which plays 
the most important role in the theory of hypergeometric series. The series 
r+1r(a1, ...,@r41; by, ..., bp} q, Z) is very-well-poised if 
(Sb) gay = apb) — G3b9 = --- = a, 10, a — gat!”, ae = qa!” 
where we fixed a square root ai! *| Following [GR], we use a more compact 
notation 


ri Vala G4,45,-.-,4r4159, 2) 


for the very-well-poised basic hypergeometric series (5.1.a). The remaining 
parameters a2,a3,5,,...,b, are excluded from the notation because they 
are determined from (5.1.b) (we assume that a; #0 for i=1,...,.r+ 1). It 
was observed in [FT, Lemma 9.2], that for 
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— a 2a 2 
=q BQ Chee an, ey — ae 


with a1, @4,Q5,...,@-41 € C and 


: r—3 
(5:.¢) z=q* with ¢= 5 tg 1) 0g as a, 
we have the following identity 


(5.1.d) v1 Wp (q?; g2%4, i qth. q@’, qs) = 
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ie lar; h}n [oa; h|n [orgijh]n [ert 2njn 


5 le! [Dt+ea-aajhln +a aryizhln — [aaln 


where q = exp(7/—1h) with h € C and 
[a3 h}n = [ola [a + Ya ---[a+n— 1h. 


Here we use the notation of Section 3, in particular, the square brackets 
[...J, are defined by (3.1.b). (We correct here a misprint in the expression 
for ¢ given in [FT], Lemma 9.2.) 

If, additionally to the assumption (5.1.c), ¢ = 1, ie., if 


Lae ie 
2 2 


(5.1.e) Q1 — 04 —- G5 — ++ — Ary = 0 
the very-well-poised basic hypergeometric series -+1 We (qe) @e"* 3, grorti: 
q’,q°°) is then balanced, cf. [GR]. 


5.2. Definition of modular hypergeometric functions. We use for- 
mula (5.1.d) to define an elliptic generalization of balanced very-well-poised 
basic hypergeometric series. Let a1,04,05,°°* ,Q@r41 be complex numbers 
satisfying (5.1.e). Let h,r € C with Im(r) > 0 and h ¢ Z+Zr. The 
corresponding modular hypergeometric function is the series 


(5.2.a) pew, (Q1; ee see, ©) — 
x [o15h, T}n T\n ; faa; hs Tn me [ar413h, Tn : [a1 + 2n]n,7 
(nlne! [L+a1—agjh,t]n [L+e1 —orgij3h,TIn [ea]h,r 


where the square brackets are defined by (1.1-f) and 
(5.2.b) lo; h, T]n = [@]ajr [2+ Yas «la +n —- le 


To avoid the problem of convergence, we shall consider only the case when 
the series (5.2.a) is terminating. To ensure this condition it suffices to 
assume that at least one of the parameters a1, 4, .-.,@r41 is a non-positive 


integer. 
The term “modular” is justified by the following theorem. 


5.3. Theorem. (i) Modular hypergeometric functions are invariant under 
the natural action of SL(2,Z) on the variables h,7, 1.€., 


h ar+b 
cer+d’cr+d 


C A) € SL(2,2Z). 


) ome r41Wr(ar; O44 +++) Br41; h, T); 


Pia, (en O64, +++) Arq; 


for all 
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(ii) If 01, 04, ..., Ar41 € Z, then function (5.2.a) is invariant under the 
natural action of Z? on the variable h, namely 


Pw (O1) 4, cay Ope; Ro mE ah, pie i ay ey Ones ee) 
for all (m,n) € Z?. 


Proof. (i) It follows from the transformation properties of the theta 
function 6;; that the elliptic number [a],,, is invariant under the trans- 
formation (h,t) — (h,7 +1) and acquires the factor exp2Val® (a2 — 1) 
under the transformation (h,r) + (2,—+). Since the quantity of elliptic 
brackets in the numerator and the denominator of each term in ,4 w, is the 
same, we only need to check that the sums of squares of the numbers in the 
numerator and the denominator are equal. The shifted factorial [a;h,T]n 


contributes to the sum of squares 
il 
o +(a+1)?+---+(a+n—-1) =na?4+n(n—-1)a+ grin — 1)(2n — 1). 


This implies that the product 


r+1 


[ax; lp ale 
poe) ee 
( ) Wier, 
contributes 
r+l1 


» (naz + n(n — l)ag —n(1 +04 — ag)? — n(n -1)(1 +a, - ee) = 
k=4 


r+l 
= n(n + ay) S > (20x 2 i =a}). 
k=4 
The terms 
(5.3.b) loushytln | [ar + Onli 
[n]n,7! [or]n,r 
contribute 


n(n + a1)(a1 — 1) + (a1 + 2n)? — af = n(n + a1)(a1 + 3). 


Thus the difference of the sums of squares in the numerator and the de- 
nominator is 


He r—-5 r—38 


2n(n + ai)(>~ Ak — 5 — ree) —all) 
k=4 


Yang-Baxter equation and modular hypergeometric functions 199 
by the balancing condition (5.1.e). 


(ii) It follows from the transformation properties of 0,; that the elliptic 
number [a];,. with integer a is multiplied by (—1)°"1 under the trans- 
formation h > h + 1. Thus we need to check that the sums of numbers in 
the numerator and the denominator of each term in ,;41w, are equal mod 
2. The shifted factorial [a;h, 7], contributes 


—1 
cial heeee CTE ee ) 


2 
The product (5.3.a) contributes 
r+1 r+1 
S> (nay — n(1 + a1 — aK)) = a n(20; — 1-1). 


The terms (5.3.b) contribute n(a, —1)+2n = n(a,+1). Now the balancing 
condition implies that 


r+1 r+1 

—3 
S| n(20%—1-1)+n(a1 +1) = 2n(S > ape (14+a1)) = —2n = 0(mod 2). 
k=4 k=4 


Invariance of ,,,w, under the transformation h —~ h+7T is verified similarly 
using formula (4.2.h). 


5.4. Elliptic 6j-symbols as modular hypergeometric functions. In 
[FT, Theorem 9.4] we identified the trigonometric 6j-symbols (with spec- 
tral parameter) with balanced very-well-poised basic hypergeometric series 
10¢9. As it was shown in Section 3, the trigonometric 6j-symbols are limits 
of elliptic 6j-symbols when Im(r) — +00. This suggests to compare the 
elliptic 6j-symbols with the modular hypergeometric functions i909. 

Let a,b,c, A, B, C be non-negative integers such that the triples (a, 6, ), 
(AMBRO CAN, C), (a, B,C) are admissible. Set 


en pe oe _ Calg sae pee! 2 ee 
Ce mee = a a rn oe ee Lae 5, 


Note that a, 3,y,o are non-negative integers. Set 
b= max(A — a, B— 8,C — 7) = max(|A — a, |B — BI,|C — 71) >0 


where the second equality follows from the admissibility of the triples 
CA Bc) at), (a, B,C). 
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Consider the elliptic 6j-symbol 


[48 Jenn 
a (GemlGreiereDs obo v=o a) 
> (Sale eee ES It 5 a eore fo 


and denote by \(A, B, C,a,b,c| z,h,7) the first non-zero summand on the 
right-hand side. This summand corresponds to g = p. 


5.4.1. Theorem. For any x € C, 


Ata | ‘ J a 

A 18 ep (x) = (A, B,C, a,b,c] 2, »T) X 19W9(a1; a4, ..., 10; :T) 
where \qw9 is the modular hypergeometric series with the following param- 
eters: Q1,Q4,Q5 are the numbers 


= Be C= 
peers a1, pee a1, peta 


so that a ts the maximal of them (and equals 1+1) and ag,...,a19 are the 
numbers 


eee ee Oa CeO +o 
a 9 ; 9 ; 5 z+ 2, 5 +2+1. 


Note that at least one of the parameters 


p-A-a p-B-B yw-C-y¥ 
Z , 2 j 2 

is a non-positive integer so that the function 10W9(Q13 G4, ..., 10; h,7) is 

given by a terminating series. The proof of Theorem 5.4.1 is rather straight- 

forward, it suffices to replace the trigonometric numbers by the elliptic 


numbers in the proof of Theorem 9.4 in [FT]. To verify balancing condition 
(5.1.e) it suffices to note that 


Q +a4+...+ a9 = 4+ 6 = day 4+ 2. 


Theorems 5.3 and 5.4.1 imply that the transformation properties of the 
elliptic 6j-symbol with respect to the standard action of S L(2, Z) 

on (h,T) are completely determined by the transformation properties 
of the monomial X. This suggests another proof of Corollary 4.4 and the 
first claim of Theorem 4.2. 


Yang-Bazter equation and modular hypergeometric functions 201 


Ifxe sf, and «+ ute € Z, then all the parameters of j9wg are inte- 
gers so that the transformation properties of the elliptic 6j-symbol under 
the standard action of Z? on h are also determined by the transformation 
properties of A. This suggests another proof of equalities (4.1.b) and (4.1.e) 
in Theorem 4.2. 


5.5. Identities for modular hypergeometric functions. In [FT], 
we used the identification of the trigonometric 6j-symbols with i09¢9 to 
derive from the properties of 6j-symbols two fundamental identities in the 
theory of basic hypergeometric functions: Bailey’s transformation and the 
Jackson summation formula. The identification of modular hypergeometric 
functions with the elliptic 6j-symbols allows to generalize these identities 
to the elliptic case. 


5.5.1. Theorem. Modular hypergeometric functions satisfy the following 
generalization of Bailey’s transformation 


(5.5.a) 10W9 (1; 24, ..., 01032, T) = 10W9(G1; Pa, ---» B10; h, T) x 


[oy + 15h, T]n [61 +1 —075h,7)n [A +1 —ag;h,T\n [a1 +1 — a7 — a8;h, Tn 
[Bi +1;h,T]n lar +1—07;h,7]n [oy + 1— a8; h, T)n [1 + 1 — 7 — 83h, T]n 


where n = —Qj9 is a non-negative integer, 
By = 2a; +1—a4 — a5 — ag = —Q) —1 +07 + O8 + 9 + O10 


Bs =Bi1—o,—04, Ps=fi-—o+5, Pe =fi—a1+% 


and { 7, Bs, Bo, Bio} ts an arbitrary permutation of {a7, Ag, A9, 210}. 


The proof of this theorem reproduces the proof of formula (9.5.a) in 
[FT] with the obvious changes. Note however that this argument, based 
on Theorem 5.4.1, yields Theorem 5.5.1 only in the case when the pa- 
rameters 01, 4,-.-,@109 are integers. (In fact, two parameters, say ag, Q9, 
may be non-integers but their sum should be an integer.) In [FR] we 
dealt with trigonometric polynomials so that an equality for all integer 
values of parameters implies the equality in general. Here we need an 
additional argument as follows. Multiplying the equality (5.5.a) by the de- 
nominators of the left and right hand sides we can reduce it to the form 
l(h,7) = r(h,T) where both sides are elliptic polynomials. Present the 
difference f(h,7) =1(h,7) — r(h,7) as a power series in variable h: 


HO) =o Ie 


n>0 
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Then each coefficient f,,(7) is a cusp form of weight n since 


h art +b 
er+d’cer+d 


f( ye Ines 


for all 


¢ :) SUVA 


and limjm(7)-++00 fn(7T) = 0. The linear space of the cusp forms of a given 
weight is finite dimensional. Since f,(7) = 0 for an infinite number of 
parameters Q),@4,...,@19, the function f, is identically zero. 


5.5.2. Theorem. Modular hypergeometric functions satisfy the following 
generalization of the Jackson summation formula 


— [aa +1; h,7]n [ar +1 -— 04 — a5;h,7]n 2 


Sot Sh Csi). 7) — eer eas 


[ay +1 — a4 — a6;h,T]n [a1 + 1 — a5 — 63h, T]n 
[a +1—ag6;h, Tn [oy +1 — 4 — a5 — 263, T]n 


where n = —ag is a non-negative integer. 


The proof reproduces the proof of formula (9.5.d) in [FT] with the 
obvious changes. 


5.6. More on elliptic numbers. It is easy to verify that all identities in 
the ring generated by trigonometric numbers [z], (considered as functions 
of h) follow from the quadratic identity (0.b). We believe that all identities 
in the ring generated by elliptic numbers [2] = [z],,, 

(considered as functions of h, 7) follow from Riemann’s theta identity of 
degree 4 mentioned in the Introduction. If true, this would yield a different 
deduction of Theorems 5.5.1 and 5.5.2 from the case of integer parameters. 

In this connection, we briefly discuss the rings of trigonometric and 
elliptic integers. The ring of trigonometric integers T is generated by the 
variables [n],n € Z subject to the relations 


(5.6.a) [0] = 0, [1] = 1, [n] =—[n] for neZ 
and the relation 
[m+n+r][r] = [m+r][n +r] — [mJ [r], 


for any m,n,r € Z. (This relation is equivalent to (0.b) via the substitution 
n=Yytz,m=y-—z,r=2z-—y.) It is easy to observe that T is the one 
variable polynomial ring Z[t] with t = [2]. Similarly, the ring of elliptic 
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integers € is generated by the variables [n],n € Z subject to (5.6.a) and the 
relation 


[m +n+r 4 2l] [m] [nr] [r] = 
=[mtn+r4]] [m4] [n+] [r+ I —[mtn+] [mtr] int r+] [l] 


for any m,n,r,l € Z. (This relation is equivalent to Riemann’s theta iden- 
tity vian =yt+2z,m=y-—2,r=2-w,l=w —y.) The structure of € 
seems to be rather involved. Observe that the quotient ring obtained by 
inverting {2] is generated by [2], [3], [4]. Indeed, setting m=n=r= 1 we 
obtain 

(21+3] = fl+1)° (0+ 3] — (+2)? 


which allows to express [21 + 3],1 = 1,2,... as a polynomial of (le << 
21+ 2. Similarly, setting m = n = 1,r = 2 we obtain 


{2} (22 + 4] = 0-41]? 2 +2] (0+ 4) — (+ 2+ 3). 


This yields an expression of {21 + 4],J = 1,2,... as a polynomial of [n],1 < 
n < 2143 divided by [2]. We do not know if [2], [3], [4] are algebraically 
independent in €. 


5.7. Remark. Some of the existing proofs of Bailey’s transformation and 
the Jackson summation formula are perfectly suitable for generalization to 
the elliptic case. In particular, the proof of the Jackson formula given in 
[S] relies entirely on a comparison of zeros and poles on both sides of the 
formula and can be reproduced in the elliptic case. Taking into account the 
fundamental role of the elliptic functions in the theory of basic hypergeo- 
metric series it is surprising that this generalization was not noticed by the 
experts. We leave it as a challenging problem to generalize various known 
identities for balanced very-well-poised basic hypergeometric functions to 
modular hypergeometric functions (for example, Bailey’s four-term 1099 
transformation for non-terminating series, etc.). 
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Combinatorics of hypergeometric functions 
associated with positive roots 


Israel M. Gelfand, Mark I. Graev and Alexander Postnikov 
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/ 


ABSTRACT. In this paper we study the hypergeometric system on unipotent 
matrices. This system gives a holonomic D-module. We find the number 
of independent solutions of this system at a generic point. This number 
is equal to the famous Catalan number. An explicit basis of T-series in 
solution space of this system is constructed in the paper. We also consider 
restriction of this system to certain strata. We introduce several combi- 
natorial constructions with trees, polyhedra, and triangulations related to 
this subject. 
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1. General Hypergeometric Systems 


In this paper we use the following notation: [ayb) = {axarrl,.. ,d} 
and [n] := (1, n}. 

Recall several definitions and facts from the theory of general hyperge- 
ometric functions (see [GGZ, GZK, GGR2}). 

Consider the following action of the complex n-dimensional torus T = 
(C*)” with coordinates t = (¢1, t2,.-. ,tn) on the space C% 


(ae) g = (x1, 22,---,2n) 7 @-t= (ie erent), 


where a; = (a1j,---,4nj) €Z", j =1,2,.. _, N and ¢% denotes t{” ...tn". 
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Definition 1.1. The General Hypergeometric System associated with the 
action of torus (1.1) is the following system of differential equations on CY 


(12) EE, =O), — i) rh: 
f= 
De a \~" 
oe I ee 
Plas J gil; <0 J 
where @ = (Q,Q9,...,Qn) € C” and ! = (,lo,...,ln) ranges over the 


lattice L of integer vectors such that l,a; + lgag+--:+lyvan =0. 

Solutions of the system (1.2), (1.3) are called hypergeometric functions 
on C% associated with the action of torus (1.1). The numbers a; are called 
exponents. 


Remark 1.2 Equations (1.2) are equivalent to the following homogeneous 
conditions 


(1.4) fet) =t° f(a), 
where ¢ =i(ti,t3,.46,) € 2 andd* — 7, 't,°2140 7. 


Remark 1.3 For a generic a system (1.2), (1.3) is equivalent to the sub- 
system, where L ranges over any set of generators for the lattice L. 


By A denote the set of integer vectors a),a2,...,an. Let Ha be the 
sublattice in Z” generated by a), a2,...,an and let m = dimH, be the 
dimension of Hy. Let P,4 denote the convex hull of the origin 0 and 
@,,42,...,an. Then Py is a polyhedron with vertices in the lattice H,. 

Let Volz, be the form of volume on the space H4 @z R such that 
volume of the identity cube is equal to 1. The volume of a polyhedron with 
vertices in the lattice H,4 times m! is an integer number. In particular, 
m! Volz, Pa is an integer. 


Theorem 1.4. The general hypergeometric system (1.2), (1.8) gives a 
holonomic D-module. The number of linearly independent solutions of this 
system in a neighborhood of a generic point is equal to m! Voly ule 


If there exist an integer covector h such that 
(1.5) ep — 1. for all tle, wv 


then we call the corresponding system (1.2), (1.3) flat or nonconfluent. 

Theorem 1.4 in the nonconfluent case was proved in [GZK]. Very close 
results were found by Adolphson in [Ad], but his technique is quite different 
from ours. In this paper we study one special case of systems (1.2), (1.3) 
when condition (1.5) does not hold. We define these systems in the following 
section. 
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2. Hypergeometric system on unipotent matrices 


Let RC Z” be a root system and R* C R the set of positive roots (see 
[Bo]). Then we can define the hypergeometric system (1.2), (1.3) associated 
with the set of integer vectors A = Rt. 

We consider the case of the root system A, in more details. Let 
€0,€1,--+,€n be the standard basis in the lattice Z"*1, The root system 
An is the set of all vectors (roots) ej; = € — €j,7 # j. Let A = Af be 
the set of all positive roots A = {e;; € An : 0<i<j< n}. It is clear that 
positive roots generate the n-dimensional lattice Ha ~ Z” of all vectors 
UV = Voeo + U1, +°°> + Unen, vi € Z such that vo + ui +--+ + Un = 0. 

By Z, denote the group of unipotent matrices of order n+1, i.e. the 
group of upper triangular matrices z = (zi), O<t<j<n with 1’s on the 
diagonal z; = 1. The n-dimensional torus T presented as the group of 
diagonal matrices t = diag(to,ti,-.-,tn), to-ti---tn = 1 acts on Zn by 
conjugation z € Z, — tzt~/, or in coordinates 


(a1) 212 Casal 


Clearly, the action of torus (1.1) associated with the set of vectors A= At is 
the same as action (2.1). Here N = ()) and 2;;, 0<i<j<n are coordinates 
mc. 

The main object of this paper is the hypergeometric system associated 
with action (2.1). Write down this system explicitly. 


Definition 2.1. The Hypergeometric System on the Group of Unipotent 
Matrices is the following system of differential equation on the space ee 
CN with coordinates z;;, 0<7<j<n 


(2.2) 
jou n 
of a 
= d ae Ce, + oy Zjk 7S, =O, y) 1s ee Or 

Of Oot edt 

Se 0<i<j<k<n, 
(2.3) OZik C, OZ jk d ae = 
where a = (a9,01,---;@n) € C”*? is a vector such that ia — 0. 


Solutions of system (2.2), (2.3) are called hypergeometric functions on 
the group of unipotent matrices. 


In order to prove that system (2.2), (2.3) is a special case of the system 
(1.2), (1.3) we need the following simple lemma. 


Lemma 2.2. It follows from equations (2.3) that 


ne ae 


z me 
(i,j): lig >O 2 (i,9): lag <0 
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Ord (EOE St <a, Ws € Z such that See ine = Sh Lik =O 
Oe 
Proof. It follows from (2.3) that 

Oi Oo? f 


Oz; OZ) OZ elas Ca, 


Now change in (2.4) all occurrences of 5=— to Oe ee We get 
the same expressions in LHS and in RHS. 


Let 2, = P,+ be the convex hull of the origin 0 and of e;;, O<i<j<n. 
The first part of the following theorem is a special case of Theorem 1.4. 
Theorem 2.3. 


(1) The hypergeometric system (2.2), (2.3) gives a holonomic D- 
module. The number of linearly independent solutions of this system 
in a neighborhood of a generic point is equal to n! Vol Py. 

(2) n! Vol P, is equal to the Catalan number 


Cue 1 (“"). 
nti\n 


3. Integral expression for hypergeometric functions 


In this section we present an integral expression for hypergeometric 
functions on unipotent matrices (see [GG1]). 
Consider the following integral 


(3.1) f(z)= ii exp (SD ea) ree, 


where the sum in exponent is over 0<i<j<n; t is a point of torus 
SO eae) i ee eT aes (Cet att 
t) ...t,0" dty/t) ...dtn/tn; and C is a real n-dimensional cycle in 2n- 
dimensional space T. 


Theorem 3.1. The function f(z) given by integral (3.1) is a solution of 
the hypergeometric system (2.2), (2.3). 


4. T-series and admissible bases 


In this section we construct an explicit basis in the solution space of 
system (1.2), (1.3). In the case of nonconfluent systems this construction 
was given in [GZK]. In this section we basically follow [GZK). 
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Recall that A = {a1,a@9,...,an}, where a; € Z". Without loss of 
generality we can assume that vectors a; generate the lattice Z”, ie. Ha = 
Z". Let y = (71, 72)---» Yn) € CN. Consider the following formal series 


yl 
(4.1) é,(2) = )- ——————_, 
x eS Ge +1; + 1) 


where x = (£1,22,.--,2N), L is the lattice such as in Definition 1.1, and 
pa We me 

Lemma 4.1. The series ®,(x) formally satisfies system (1.2), (1.3) with 
a= Dy, jj. 


For a fixed vector of exponents a = (a1,...,@) the vector y = 
(71,---,W) Tanges over the affine (N — n)-dimensional plane (a) = 
{(71)---, YN) : 257747 = o}. In this section we construct several vectors 


+ such that all series ®,(z) converge in a certain neighborhood and form a 
basis in the space of solutions of system (1.2), (1.3) in this neighborhood. 
A subset Z € [N] is called a base if vectors a;, j € T form a basis of 
the linear space H, @ R. So we get a matroid on the set [N]. Let Az be 
the n-dimensional simplex with vertices 0 and a;, j € TZ. 
Let Z be a base. By l(a, Z) denote the set of y € II(@) such that 7; € Z 
for j ¢ Z. It is clear that for every | € L (see Definition 1.1) &,(z) = ©4412. 
The following lemma was proven in [GZK]. 


Lemma 4.2. Let Z be a base. Then |I(a,Z)/L| =n! Vol(Az). 


Definition 4.3. We call a base Z € [N] admissible if the (n — 1)- 
dimensional simplex with vertices a;, 7 € Z belongs to the boundary 0P4 
of the polyhedron P,. In this case the simplex Az is also called admissible. 


Remark 4.4 If vectors a; satisfy condition (1.5) then all bases are admis- 
sible. 


Let B = {b,,b2,...,b6v—-n} be a Zbasis in the lattice L. We say that 
a base Z is compatible with a basis B if whenever | = (l,,...,4w) € L such 
that 1; > 0 for j ¢ Z then I can be expressed as 1 = > Axby, where all Ax 2 
0. Clearly, the set Ip(a,Z) = {7 € Ha, T):y¥=DAnbk, where 0<rA;,<1} 
is a set of representatives in I(a,Z)/L. 

Let y, =2'*, K=1,2...,.N—n. 
Proposition 4.5. Let an admissible base I be compatible with a basis 
B. Then for all y € Ip(a,Z) the series @,(x) is of the form ®,(x) = 
a1 3~_, c(m)y™, where the sum is overm = (m,,...,MN-n), Mz =O. The 
series ).c(m)y™ converges for sufficiently small |yx|. 


Proof. Leta. — (bei,---,OKn) Ee 1 At By definition, 
@,(x) = 27 Decne. where c(m) = I; T(95 + Sop Mebeg + 1)1,m= 
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(m,...;7n_n) € ZY". Let y € Ig(a@,Z). Then V7 +> 2, mde; +1 € Z, 
for 7 ¢ 2. Hence, if cim)e=f 0! thenty, ay mb, -- I Oe 7. 
Since Z is compatible with B, we can deduce that c(m) 4 0 only if m; > 0, 
k =1,...,N—n (see details in [GZK]). Convergence of the series 5> c(m)y” 
follows from the next lemma. 


Lemma 4.6. Let c(m) = [],;T(uj(m) + 9 + Ue a — (Cth consis, 
my = 0, where py; are linear functions of m such that ¥> u;(m) = sym, + 
++8,M,, Sp > 0. Then |e(m)| < Rey ...c™ for some positive constants 
Lear oan eR 


It is not difficult to prove this lemma using Stiltjes formula. Thus, by 
Proposition 4.5 for every admissible base I we have n! Vol(Az) series ©,(z), 
y € IIp(a,Z) with nonempty common convergence domain. 


Remark 4.7 It can be shown that if 7 € II(a,Z), where Z is not admissible, 
then ®,(x) diverges. 


Recall that P,4 is the convex hull of 0 and Gy, a 2h. 


Definition 4.8. The set of bases © is called a local triangulation of P, if 


(1) Uzee Az = Pa; 
(2) Az, M Az, is the common face of Az, and Az, for all Z;,Z> € O. 


We call such triangulation © local because all simplices Az, I € © 
contain the origin 0. 
Remark 4.9 Note that if © is a local triangulation then all bases J € O 
are admissible 


Definition 4.10. A local triangulation © is called coherent if there exist 
a piecewise linear function ¢ on P, such that ¢ is linear on simplices Az, 
Z € 0 and ¢ is strictly convex on P,. 


Lemma 4.11. There exists a coherent local triangulation Of Px 


Lemma 4.12. Let © be a coherent local triangulation of Pa. Then there 
exists a basis B of H4 such that B is compatible with every base I in @. 


Theorem 4.13. Let © be a coherent local triangulation of Pa; and B= 
{Oj 02.2 

...,bn—n} @ basis such as in Lemma 4.12. Let yx = z°*. Then for every 
7 € Ip(a,Z), T € © the series &,(x) is equal x7 times a series of variables 
Yk, which converges for sufficiently small lyz|. If exponents a1, 02,..., Am 
are generic then all these series ®,(x) are linearly independent. 


Hence, for generic a = (a, a2,...,Qn) we constructed n! Vol(P.) inde- 
pendent solutions of system (1.2), (1.3), which converge in common domain. 
Therefore, by Theorem 1.4, these series form a basis in the space of solutions 
of system (1.2), (1.3). 
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5. Admissible trees 


In this section we describe admissible bases in the case of the hyperge- 
ometric system (2.2), (2.3). 

It is well known that a subset Z C {(i,j) : 0<i<j<n} is a base in the 
set of positive roots A = A? if and only if Z is the set of edges of a tree Tz 
on [0, n]. 


Definition 5.1. A tree T on the set [0,7] is called admissible if there are 
no 0<i<j<k<n such that both (i,j) and (j,k) are edges of T. 


Proposition 5.2. A subset ZC {(i,9): 0<i<j<n} is an admissible base 
in A= At if and only if Tz is an admissible tree. 


Lemma 5.3. n! Vol Az = 1 for any base T. 


Therefore, by Lemma 4.2 |II(a, Z)/L| = 1 and Proposition 4.5 for every 
admissible tree T we have a series &y(z) = ®,(z), where y € II(a,Z), 
T =Tr. The series ®7(z) converges in some domain and presents a solution 
of the system (2.2), (2.3). There exists a formula for the number of all 
admissible trees on the set [0, n}. 


Theorem 5.4. The number F,, of admissible trees on the set of vertices 


(0, n] is equal to 
n+1 
a ("*") Kk. 
2 nal) ee k 


The proof of this formula is given in [Po]. 
The first few numbers of F,, are given below. 


n 0 ul Z 3 4 5 6 7 
ee if if 2 7 36 246 2104 21652 


6. Standard triangulation of P,, 


Recall that P, is the convex hull of 0 and e;;, O<1<j<n. 

In this section we construct a coherent triangulation of the polyhedron 
P,,. This will give us an explicit basis in the solution space of system (2.2), 
(2:3) 

Let T be a tree on the set [0,]. We say that two edges (i, j) and (k, /) 
in T form an intersection if i<k<j <i: 


Definition 6.1. A tree T on the set [0, n] is called standard if T is admis- 
sible and does not have intersections. The corresponding base Z C {(i, J) : 
0<i<j<n} is also called standard. 
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FIGURE 6.1. Standard trees. 


Example 6.2. All standard trees for n = 0,1,2,3 are shown on Figure 
6.1. 


Theorem 6.3. The set ©, of standard bases forms a coherent local trian- 
gulation of the polyhedron P,,. 


Theorem 6.4. The number of standard trees on the set [0,n] is equal to 


the Catalan number 
C, = 1 (*") 
n+1\n 


As a consequence of these two theorems we get Theorem 2352). 
Proof of Theorem 6.4. Construct by induction an explicit 1-1 correspon- 
dence 7, between the set ST, of standard trees on [0,n] and the set BT,, 
of binary trees with n unmarked vertices y, : ST, —~ BT. 

If n = 1 then y% maps a unique element of ST) to a unique element of 
Bir 

Let n > 1. Every standard tree T € ST,, has the edge (0,n). Delete 
this edge. Then T splits into two standard trees T; € BT; and Ty € BT), 
k+1+1 =n on the sets [0,k] and [k+1,n]. Let as define ~n(T) as the 
binary tree whose left and right branches are equal to wo. (T1) and u(T2) 
correspondingly. See the example in Figure 6.2. 

It is well known that the number of binary trees is equal to the Catalan 
number (e.g. see [SW)). 

Now prove Theorem 6.3. 


Proof of Theorem 6.3. Recall that €0,€1,---,€, 1s the standard basis in 
Z"*); and ei; = € — €;. 
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Ws 
———2- 
© 2 A B 2 Ss 


FIGURE 6.2. Bijection between standard and binary trees. 


Let P, c Zt! @R denote the cone with vertex at 0 generated by all 
positive roots e;;,7 <j. Let Ar denote the simplicial cone generated by 
ei;, (i,7) € Z, where T is a base (the cone over the simplex Az). 

First, prove that the collection of cones Ar, where TZ range over all 
standard bases, is a conic triangulation of P,,. Then it follows that ©, is a 
local triangulation. It is not difficult to show that the cone P, is the set. of 
v = (v0, 1,--+,Un) € R®*? such that 


(6.1) vo tui ++--+u; = 0, ee ee 
(6.2) Vo tui tee + Un = 0. 


We must show that every generic point v subject to (6.1), (6.2) can be 
uniquely presented in the form 


(6.3) v= > pizeig, pig 20, 
(ij) ET 


for some standard base Z. Prove it by induction on n. 

Let v’ = (vp, U},---;Un—-1) € R®” be a vector such that vi = i,t = 
Onipeeer 2 and! 7 ae— dae, Phen. eur By induction we 
may assume that v’ is expressed in the form 


v= iD Pig eis Piz 2 9, 
(77) ET’ 


for a standard base Z’ C {(i,j) : 0<i<j<n-1}. 

Let i, <ig<...<ig be all vertices of T’ = Tz connected with the vertex 
n—-1 in T’. 

Consider two cases. 

1. Un_1 > 0. Define Z = Z’U{(n-1,n) }U{(ix, 2) : k € [s]}\{(%, n—1) : 
k € [s]}. And pi; = pj; for 0<i<jsn—2; pin = Piyn—1 for k € [s]; 
Pn—in = Un—-1. Then we get expression (6.3) for v. 

2. Un_1 < 0. Then —Un < op=1 Pizn—-1- Let t be the minimal integer 
O<t<s such that )~,— Pi,n-1 2 —Un- Then define Z = DOG k 
i} \ {(insn—1) 2 € [e-}. And pig = pay for OSi<j<M—25 ign = Plynt 
fork € ty Pin = = Peel — Un; Pipn-1 = Pinat for k € Eel si; 
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Pien—1 = — > kat41Pign-1 — Un—1- Then we get expression (6.3) for v. 
Therefore, ©, is a local triangulation. 

Prove that ©, is coherent triangulation (see Definition 4.10). We must 
present a piecewise linear function ¢ on P, such that ¢ is linear on all 
simplices in ©, and ¢ is strictly convex on Py. 

It is sufficient to define ¢ on vertices of P,. Let 6(0) = 0 and ¢(6;) = 
(i — 7)?. It is not difficult to show that such ¢ satisfy the condition of 
Definition 4.10. 

Now we can complete the proof of Theorem 2.3. 


Proof of Theorem 2.3. 

The first part of Theorem 2.3 is a special case of Theorem 1.4. 

The second part follows from Theorems 6.3, 6.4 and Lemma 5.3. 

To conclude this section we present a construction of another coherent 
triangulation of P,,. 

Let T' be a tree on the set [0,n]. We say that two edges (7, 7) and (k,l) 
in T' are enclosed if i<k<Il<j. 


Definition 6.5. A tree T on the set [0,n] is called anti-standard if T 
is admissible and does not have enclosed edges. The corresponding base 
ZC {(i,j) : O0Si<j<n} is also called anti-standard. 


Theorem 6.6. The set of anti-standard bases forms a coherent local tri- 
angulation of the polyhedron P,. 


The proof of this theorem is analogous to the proof of Theorem 6.3. 


Corollary 6.7. The number of anti-standard trees on the set [0, n] is equal 
to the Catalan number C,,. 


7. Coordinate strata 


Let Z, be the group of unipotent matrices 244, USIS 75, 2 = | (see 
Section 2). 

Consider a subset S C {(i,j) : 0<i<j<n}. By Zs denote the set of 
all z = {2} € Z, such that z;; 4 0 if and only if (7,7) € S. We call 
Zs coordinate strata in the space Z,. Let Zs ~ CIS! be the closure of the 
stratum Zo. 

We can construct two sheaves of hypergeometric functions on the mani- 
fold Zs, where S c {(i,j): 0<i<j<n}. First, the sheaf Resg of restrictions 
of hypergeometric functions on Z, to the manifold Zs. Second, the sheaf 
Sols of solutions of the hypergeometric system (1.2), (1.3) associated with 
A= As = {ei; : (i, 7) € S} (equivalently, associated with action (2.1) of 
torus on Zs). 

The question is: when do these two sheaves coincide? 


Definition 7.3. Let P = {bo, b1,...,bn} be a partially ordered set (poset) 
such that if b; <p b; then i < j. Consider the set Sp = (59), 2b by 
We call this set associated with poset P. 
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Theorem 7.4. Let S= Sp be the set associated with a poset. Then sheaf 
Resg coincides with sheaf Sols for generic exponents ao,...,Qn, >, a = 0. 


Remark 7.5 By Theorem 1.4 the dimension of Sols in a neighborhood 
of a generic point is equal to m! Volz (s) P(S), where H(S) is the lattice 
generated by e;;, (2,7) € S, m = dim Hg, and P(S) is the convex hull of 
the origin and e;;, (i,j) € S. 


Proposition 7.6. A set S Cc {(i,j) : 0<i<j<n} is associated with a poset 
P if and only if there exists a cone C with verter at 0 such that S = {(t < 
9) a C}. 

Proof. A set S is associated with a poset if and only if S satisfies the 
following transitivity: if (2,7), (j,k) € S then (i,k) € S. The set S = 
{(i < j) : ei; € C} satisfies transitivity because if e;;,e;x € C then ex, = 
ei; + ej € C. Inversely, let C be the cone generated by all e;;, (7,7) € S. 
If S satisfies transitivity then S = {(i < j) : ex; € C}. 


Now we can prove Theorem 7.4 


Proof of Theorem 7.4. Clearly, Resg is a subsheaf of Sols. Suppose for 
simplicity that e;;, (i,j) € S generate Z”. The dimension of the sheaf Sols 
at a generic point is equal to n! Vol(P(S)) (see Remark 7.5). Hence, it is 
sufficient to prove that the dimension of Ress at a generic point is greater 
than or equal to n! Vol(P(S)). 

Let © be a coherent local triangulation of P(A). It follows from Propo- 
sition 7.6 that © extends to a coherent local triangulation 0’ of P,. Con- 
sider n! Vol(P(S)) T-series ®(z) on Z,, where y € H(a,Z), Ze OC oO’. 
By Theorem 4.13 these series are linearly independent and have a com- 
mon convergence domain. Then restrictions of these series to Zs give 
n! Vol(P(S)) independent sections of the sheaf Resg in some neighborhood. 
Therefore, Ress = Sols. 


8. Face strata 


Describe faces of the polyhedron Pp. 
Let I,J Cc [0,n], IN J =9@. Let Sj be the set of all @, 7), OS?<7sn 
such that i € I and j € J. 


Proposition 8.1. Faces f of the polyhedron P,, such that 0 ¢ f are in 1-1 
correspondence with sets Syz. And (1,9) € Sry whenever ej; is a verter of 
the corresponding face f. 


Clearly, we may assume that min(IUJ) € J and max(IUJ) € J (if Sy 
is nonempty). 

Construct a coordinate stratum associated with a face f of P, 0 ¢ f. 
Let S = S; 7. By Zy; 7 denote the stratum Zs (see Section 3). We will call 
such strata face strata. 

Note that condition (1.5) holds for vectors e;;, (2,3) € Su; because all 
such e;; belong to a supporting hyperplane of the corresponding face f. 
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Definition 8.2. The Hypergeometric System on Z1, is the hypergeometric 
system (1.2), (1.3) associated with the set of vectors A = {e;; : (t,7) € Sry}. 
Solutions of this system are called Hypergeometric Functions on Z1). 


Remark 8.3 Let 0<p<n, I = {0,1,...,p}, and J = {p+1,p+2,...,n}. 
Then Z;7 is the space of rectangular matrices z = {z;;}, i € [0,p], 
j € [p+1,n]. The hypergeometric system on Z 7, is also called the Hyperge- 
ometric System on the Grassmannian Gr+1p41. This system was studied 
in the works [GGR1, GGR2, GGR3]. 


It is clear that the set S = S7j is associated with a poset (see Definition 
7.3). Therefore, by Theorem 8.4, the sheaf Ress coincides with the sheaf 
Sols of hypergeometric functions on Z;,7 (for generic a). 

We will find the dimension of this sheaf in a neighborhood of a generic 
point. Denote this dimension by D; ;. In other words, D7 is the num- 
ber of independent solutions of the hypergeometric system on Zy, in a 
neighborhood of a generic point. 

Let Pz be the convex hull of 0 and e;;, (4,7) € S7z. Let Hyz be the 
sublattice generated by e;;, (i, 7) € S77, and m = dim H;. By Theorem 1.4 
the number Dy; is equal to m! Voly, ,(Prz). 

We present an explicit combinatorial interpretation of this number DJ. 


Definition 8.4. 


(1) A word w of type (p,q) is the sequence w = (207), 409, 306, pg): 
wy € {1,0} such that |{r: w, = 0}| =p and |{r: w, = 1}| =q. 
(2) Let w = (wi, we,...,wptq) and w! = (wi, we,..., why.) be two 


words of type (p,q). We say that w’ is exceeds w if wi +---+w!. > 
ise tet. fOU- alles — le ee) ta, 


(P.q) 


FIGURE 8.1. The word w’ exceeds the word w. 
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We can present a word w of type (p,q) as the path m = 
(Go, eee ee ceecuchthat re —s(ee7.)eior alls = 0, 1 yp-#g, 
where i, (correspondingly, j,) is the number of 0’s (correspondingly, 1’s) in 
W1,W2,...,Ws-. See example for (p, g) = (10,7) on Fig. 8.1. 

Clearly, a word w’ exceeds a word w if and only if the path x’ corre- 
sponding to w’ is above the path m corresponding to w. (See Fig. 8.1.) 

Let a = min] and b = maxJ. Then D,,; # 0 if ond only if a < b. 
Suppose that a < b, I = {a}UI’ and J = {b}UJ’, where I’, J’ c [a+1, b—1], 
Hop o—eweta — pl iqie—mgeancde My = tiperjc:-- < toi}. 
Associate with the pair (I, J) the word wyz = (w1,..., Wp+q) of type (p,q) 
Such thatw.— Oil t,.6 and w,—=1ift,eJ for allr =1,2;..., pte: 


Theorem 8.5. The number D,, is equal to the number of words w’ of type 
(p,q) which exceed the word w = wyy. In other words, D;z is the number 
of paths x’ from (0,0) to (p,q) such that 1’ is above the path m = mJ 
corresponding to wy . 


Corollary 8.6. Let [ ={0/24,..., 2k} and J = {1,3,5,...,2k-+1} then 
Dy, is equal to the Catalan number Cx. 


Proof. Words w!' = (w{,w,---,W5,) of type (k, k) which exceed the word 
w = (1,0,1,0,...,1,0) are called Dyck words. It is well known (see e.g. 
[SW]) that the Catalan number C; is equal to the number of Dyck words. 


9. Standard triangulation of Py ; 


Let I,J c [0,n], IN J = be two subsets such that min(IUJ) € I and 
max(IUJ) € J (see Section 8). 

Recall that Py = Conv(0, e;; : (i,7) € S77). In this section we present 
a coherent local triangulation of the polyhedron P; ; and prove Theorem 8.5. 


Definition 9.1. Let T be a tree on the set [ UJ. We say that T' is 
of type (I,J) if for every edge (7,7) in Ti ¢ J andj € J. The base 
Lc {(i, 7) : 0<i<j<n} corresponding to T is also called of type G7) (Wo 
not confuse Z with I.) 


Clearly, all trees of type (I, J) are admissible (see Definition 5.1). 


Theorem 9.2. The set Q7, of all standard (see Definition 6.1) bases of 
type (I, J) forms a coherent local triangulation of the polyhedron Pry. 


The proof of this theorem is essentially the same as the proof of Theo- 


rem 6.3. 
It is clear that Dy = m! Vol( Py) is equal to the number of all standard 
bases (trees) of type (I,J). Prove that this number coincides with the 


number given by Theorem 8.5. 


Theorem 9.3. Let |I| = p+1 and |J| = q+1. Then the number of all 
standard trees T of type (I,J) is equal to the number of words w’ of type 
(p,q) which exceed the word w = wyy. 
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Proof. Let Dy; be the number of all standard trees of type (J, J) and Dy; 
be the number of words w’ of type (p,q) which exceed the word w = wyj 
(we use the same notation as in Theorem 8.5). 

We prove that D7; = iD by induction on p+q. Obviously, this is true 
for p = q = 0. Let d be the minimal element of J_and c be the maximal 
element of IJ such that c < d. Let I =1\ {c} and J = J \ {d}. Prove that 
ifp+q> 0 then 


(9.1) Dyj = Dj; + D7;- 


Every standard tree of type (J, J) has the edge (c,d). In every such 
tree either c or d is an end-point. The first choice corresponds to the term 
D;, and the second choice corresponds to the term D,; in (9.1). 

The numbers Dy, also satisfy the relation (9.1). The first term corre- 
sponds to the case when the word w’ starts with 0 and the second term to 
the case when w’ starts with 1. Therefore, we get by induction D;z = Dy,. 


Theorem 8.5 is a corollary of Theorem 9.3. 


10. Examples 


In this and the next sections we present several examples which il- 
lustrate the notions introduced in the paper and show the direction for 
following study. 


10.1. Case n = 2. 

In this case the solutions f of the system (2.2), (2.3) are functions of 
variables 201; 202) 212- 

Let 6; = (a2 — 2a9) and Bo = 3 (2a9 — ao). piers of homogeneous 
conditions (1.4) we can write f (201, 202, 212) = ze zh F(y), where y = 
=—%.. Now system (2.2), (2.3) is equivalent to the following equation on 


201212" 


F(y). 


dF d d 
(10.1) aa = (v5, - 61) (v5 = ba) F. 


This is the degenerate hypergeometric equation and its solutions can be 
written in terms of the degenerate hypergeometric function ; F, (see (BE}). 
This system has two dimensional space of solutions, which is compatible 
with the fact that Cy = 2. 


10.1. Upper triangular matrices. 

Let I = {0,2,...,2n} and J = {1,3,...,2n+1}. It is natural to iden- 
tify the space Z; sith the space of sill: upper triangular matrices with 
arbitrary elements on the diagonal. Consider the hypergeometric system 
on Z;3. We call this system the hypergeometric system on upper triangular 
matrices. 
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This system has the same dimension C,, of solution space as system 
(2.2), (2.3) (see Corollary 8.6). But it is nonconfluent unlikely system 
(2.2), (2.3). If fact, system (2.2), (2.3) can be obtained as a limit of the 
hypergeometric system on upper triangular matrices. For example, if J = 
{0, 2, 4\ and J = {1,3,5} then the corresponding hypergeometric system on 
Zr, can be reduced to the Gauss hypergeometric equation. And equation 
(10.1) is a limit of the Gauss hypergeometric equation. 


11. Concluding remarks and open problems 


11.1. Characteristic manifold. 

We do not prove Theorem 1.4 here. There exists a proof of this theorem 
generalizing the proof from [GZK] for nonconfluent case. 

This proof is based on the consideration of characteristic manifold Ch 
for system (1.2), (1.3). The characteristic manifold for system (1.2), Ge3) 
is the submanifold in the space CY x C% with coordinates (z,£), 2 = 
(i, een) 6 = (61,--- 567) setven by the following algebraic equations. 


N 
S| 045256; = 0, (i= ee Le 
j=1 


Ge = lige if YG =0; 


gj: 4,>0 gilj;<0 4 
i x 
[lev-0 if Due Ds 
jl; >0 fily>0 jij <0 
where | = (11, lo,-..,1n) ranges over the lattice L of integer vectors such 


that l,ay + leag +-:-+ + lnan = 0. 

Then system (1.2), (1.3) is holonomic if dimCh = N. The number of 
independent solutions at a generic point is equal to degree of Ch along the 
zero section {(0,€) : € € C%} (see [Ka)). 


11.2. Other root systems. 

We can define (see Section 2) the hypergeometric system for arbitrary 
root system R. It is interesting to find analogues of all results in this paper 
for other root systems. 

Let Pp+ be the convex hull of 0 and all positive roots r € Rt. Then 
by Theorem 1.4 the dimension of the system at a generic point is equal to 
D(R) =n! Vol(Pr+), where 7 is the dimension of R. These numbers D(R) 
can be viewed as a generalization of the Catalan numbers for arbitrary root 
system. 


11.3. Discriminant and Triangulations of Ros 

We can associate with system (2.2), (2.3) the discriminant D,, (z). The 
discriminant D,(z) is a polynomial of z = (zi3), OSi<j<n such that 
D,(z) = 0 if and only if there exists (z,€) € Ch such that € # 0, where Ch 
is the characteristic manifold for system (22),-(2:3): 
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It is an interesting problem to find an explicit expression for D,(z) 
and describe all monomials in D,(z). The Newton polytope S,, for D,(z) 
is called Secondary polytope. Vertices of S, correspond to coherent local 
triangulations of P, (cf. [GKZ]). 

In Section 6 we constructed two coherent local triangulations of Py. 
The important problem is to find all such triangulations. Analogously, 
one can define discriminant D; ;(z) associated with face strata Zr (see 
Section 8). Vertices of the Newton polyhedron for D;7(z) correspond to 
coherent triangulations of Py; ;. (Note that all triangulations of Py; are 
local.) How to describe triangulations of P; 7? 

The special case of this problem for the pair (J, J) such as in Remark 8.3 
(the hypergeometric system on the grassmannian) is connected with trian- 
gulations of the product of two simplices A? x A%, p+ q=n+1. In this 
case Dy, is the product of all minors of (p+1) x (q+1)-matrix z (see [GKZ], 
Oi eA, BVA), 
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Local invariants of mappings 


of surfaces into three-space 


Victor V.Goryunov * 


Abstract 


Following Arnold’s and Viro’s approach to order 1 invariants of 
curves on surfaces [1, 2, 3, 20], we study invariants of mappings 
of oriented surfaces into Euclidean 3-space. We show that, besides 
the numbers of pinch and triple points, there is exactly one integer 
invariant of such mappings that depends only on local bifurcations 
of the image. We express this invariant as an integral similar to 
the integral in Rokhlin’s complex orientation formula for real alge- 
braic curves. As for Arnold’s J* invariant [1, 2, 3], this invariant 
also appears in the linking number of two legendrian lifts of the 
image. We discuss a generalization of this linking number to higher 
dimensions. 

Our study of local invariants provides new restrictions on the 
numbers of different bifurcations during sphere eversions. 


In [17, 18] Vassiliev introduced the notion of a finite order invariant 
of knots. Finite order invariants can be defined for mappings in more 
general settings so long as the discriminant has codimension 1 in the 
function space. In [1] Arnold defined three invariants of order 1 for plane 
curves. These invariants have a local nature: they do not distinguish 
different components of the same top strata of the discriminant. Recently 
Viro generalized Arnold’s invariants to the case of curves on surfaces {19}. 
Viro’s generalization is based on Rokhlin’s complex orientation formula 
for real algebraic curves [12, 13]. 

In this paper we consider mappings of oriented surfaces to Euclidean 
3-space. We study special order 1 invariants which we call local. The 
notion of locality is slightly more rigid than the one used by Arnold: we 
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require that the values of an invariant get the same increment once the 
images of the mappings experience the same local bifurcation. 

We give a complete list of integer and mod 2 local invariants of generic 
mappings and of generic immersions (Sections 2 and 3). For example, 
we show that, besides the numbers of pinch and triple points, there is 
exactly one more integer local invariant, which we call J3, of generic 
mappings of surfaces into R®. Our invariants provide new restrictions 
on the numbers of bifurcations during sphere eversions (subsection 3.2). 

In Section 4 we relate J3 to a local invariant I f which is very similar 


to the integral form of Rokhlin’s complex orientation formula [14, 19, 20]. 

Arnold found a formula for his direct selftangency plane curve invari- 
ant J+ using linking numbers of corresponding legendrian curves [2, 3]. 
In Section 5 Ir is expressed in terms of linking numbers in ST*R? of two 
varieties homeomorphic to the source surface. One of the varieties is a 
modified legendrian lift of the image of a generic mapping, the second 
one is the ‘negative’ of the first. 

In Section 6, following [2, 3] and Section 5, we introduce two linking 
numbers for a generic immersed hypersurface in R”. These numbers 
are local invariants dual to the direct and inverse selftangencies of a 
hypersurface. 

Section 7 contains proofs of all theorems of the previous sections. 


The author is very grateful to The University of Georgia, Athens, 
and to Matematisk Institut, Aarhus Universitet, the institutions where 
the work was done, for their kind hospitality and support. 


1 Generic degenerations 


Locally, the image of a generic mapping of a fixed closed surface M to 
R? is either a smooth sheet, or transversal intersection of either 2 or 3 
smooth sheets, or a Whitney umbrella (the image of (2, y) + (x, y?, ry)) 
(see Fig.1). Mappings with more complicated images form a discriminant 
hypersurface A in the infinite-dimensional space Q of all C° maps M > 
R?. 

The discriminant subdivides 2 into connected components. A numer- 
ical invariant is a way to assign numbers to each of these components. 

Moving along a generic path in 2, we watch the jumps, aS we pass 
the discriminant, of the values of an invariant. We say that our invariant 
2s local if every jump is completely determined by the diffeomorphism 
type of the local bifurcation of the image at the instant of crossing of the 
discriminant. 
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Figure 1: Local singularities of images of generic maps of surfaces into 
3-space 


Remark. Arnold’s plane curve invariant St [1, 2, 3] is not local in our 
sense since the coorientation of the corresponding stratum of the dis- 
criminant involves global information about the image. 


1.1 The top strata 


There are 7 types of local events on the image that take place at generic 
points of A, i.e. occur along generic curves in 2 [11, 10, 8] (Figs. 2 and 
a): 


(E) elliptic tangency of two smooth sheets; 
(H) hyperbolic tangency of two smooth sheets; 


(T) (for ‘triple’), tangency of the line of intersection of two smooth 
sheets to a third smooth sheet (all the three sheets are pairwise 
transversal); 


(Q) (for ‘quadruple’), four smooth sheets intersecting at the same point; 
(C) (for ‘cup’), a smooth sheet passes through a pinch point; 


(B) birth of a bubble with two pinch points joined by an interval of 
selfintersection; 


(K) (for ‘cones’ in Russian), the hyperbolic version of B. 


A local coordinate form for B and K is (z,y) + (2, y?, ya’? —A)) 
(the real parameter A increases from the left to the right in Fig.3 being 


zero in the middle). 
The seven bifurcations define seven top (i.e. top-dimensional) strata 


of the discriminant. 
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Figure 2: Local bifurcations of images in generic 1-parameter families of 
mappings involving only smooth sheets 
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Figure 3: Local bifurcations of images in generic 1-parameter families of 
mappings involving pinch points 


C 


1.2 Coorientation of the strata 


In order to assign a jump of an invariant to a stratum, we will need to 
coorient the stratum (unless we work with a Zo-invariant). 

There is a natural way to coorient 5 of our 7 strata. ‘This corresponds 
to moving from the left to the right in Figs. 2 and 3. Namely, we say 
that a passing through the stratum is positive if the passing gives birth 
to something new on a generic image: 


e FE, a new circle of selfintersection; 
e T, two new triple points; 

e C, one new triple point; 

e B, K, two new pinch points. 


In all these cases, except K, this is equivalent to the appearence of a 


new 2-cycle on a generic image. 
In the two remaining cases, H and Q, the similar procedure fails: 
the initial and final pictures of Fig.2 in each of the two series are locally 


diffeomorphic. 
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1.3. Refinement for oriented surfaces 


From now on we assume the source surface M oriented. This allows us 
to refine the stratification of the discriminant. 

Let us fix orientations of M and R3. We obtain a canonical coorien- 
tation of the image at immersive points requiring that the frame (normal 
of the coorientation, the image of a positive frame of M) is a positive 
frame in R3. 

We now add the induced coorientations of the sheets in all the 7 bifur- 
cations of the previous subsection. This splits them in 20 subcases listed 
below (see Fig.4 for 15 of them). Considering a 2-sphere that consists of 
pieces of several smooth sheets (with their own coorientations), we call 
a piece positive if it has the outer coorientation. The outer coorientation 
is also called positive, the inner one is called negative. 

We split the strata: 


© FT, Q in 7,5 = 0,1,2, 7,9 = 0,1,2,3, 9 = 2,3,4 Here 7 
is the number of positive pieces of the appearing sphere (there is 
the vanishing tetrahedron in the Q-cases which has 4 — j positive 
pieces); 


e Hin H* and H™: the sheets, at the tangency point, have coinciding, 
respectively opposite, coorientations; 


e C in C4 a, = +, —: for the appearing cup-shaped 2-sphere, a 
and f are the signs of the coorientations of the lateral (having the 
pinch point) and bottom pieces respectively; 


e B in Bt and B-: the superscript is the coorientation of the ap- 
pearing sphere; 


e K in K* and K~: the index is the coorientation of the local ‘tubu- 
lar’ part before the bifurcation. 


We continue with the coorientation of the discriminant. 


Q. We coorient the Q* and Q?® strata in the direction of increase of the 
number of positive faces of the local tetrahedron (Fig.5). But there is 
still no local (in the sense introduced above) way to distinguish between 
the two sides of the Q? stratum. 


H. To coorient H~ we consider the two points of M at which we get 
tangency for the degenerate mapping. We say that the crossing of the 
stratum is positive if the relative motion of the images of the two points 
occurs opposite to the coorientations of the sheets (Fig.5). 
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Figure 4: Local images of oriented surfaces to the positive sides of the 
top strata of the discriminant 


Once again, there is no way to coorient H* by only local means. 


In what follows the top strata of the discriminant are always taken 
with the coorientations introduced in this section. C-aatidel! “wane. 10t 
cooriented. 


2 Lists of local invariants 


2.1 Local invariants in terms of jumps along the 
top strata 


Given an invariant J of generic mappings, we can enlarge its domain OQ\A 
to include generic points f, of a cooriented top stratum bea AG) 
(17, 18]. For this we take two points, f,pond= j=) "closestowforin ©, 


to the positive and negative sides of S' respectively. We'set TG.) = 
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Figure 5: Positive directions of bifurcations Q*, Q? and H~ 


I(f4) — I(f-). If I is a local invariant, the value obtained does not 
depend on a particular choice of generic f, € S. 


Definition 2.1 s(I) = I(f;) is the jump of the local invariant J along 
the stratum S. 


To assign a jump to a local Z)-invariant we do not need any coorien- 
tation of S. 

On the other hand, a local invariant J on 2 \ A is defined, up to an 
additive constant, by its jumps along the 20 strata of the discriminant. 
To calculate the value J(f) in this situation, we need to: 


(1) choose the value of J on a distinguished generic mapping fo; 
(2) join fo and f by a generic path y in 2; 


(3) calculate the indices of intersection y MS of + with each of the 20 
strata S; 


(4) set I(f) = 2(YNS)s(1) + I(fo). 


Of course, this must be independent of the choice of y. Since Q is 
contractible, the independence is equivalent to vanishing of the increment 
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of I (counted in the above way) along any small loop in Q around the 
set of nongeneric points of A. This set has codimension 2 in 2. 

To shorten the notation, we will write a local invariant as a linear 
combination of the strata with the coefficients equal to the jumps. For 
the moment the invariants are considered up to the additive constants. 

Checking of the independence condition (which is a rather routine 
study of generic 2-parameter families of mappings) provides: 


Theorem 2.2 The space of integer local invariants of smooth mappings 
of an oriented surface into R? is 3-dimensional. The following are basic 
invariants: 


(el 2 eC, 
(2, 2 ls 
(3) Ig = EP — E+ H-+T+ Ctr +Ct + Bt +k. 
Here T,C,B,K are the sums of all the corresponding 4 or 2 substrata. 


Of course, the best choice of the additive constants for the first two 
invariants is the vanishing on an embedding of the surface. Then Ff 
is the number of triple points and J, is the number of pairs of pinch 
points of the image of a mapping. We fix this normalizations of these 
two invariants for the rest of the paper. We will make comments on the 
constant for Js in subsection 4.4. 


Definition 2.3 Tangency of two smooth sheets of the image that have 
coinciding (resp. opposite) coorientations is called direct (resp. inverse) 
selftangency . 


Difference of values of Iz (more precisely, of 213 —/,) on two mappings 
is a modification of the number of opposite selftangencies in a generic 
homotopy between these mappings. 


Theorem 2.4 The space of mod 2 local invariants of smooth mappings 
of an oriented surface into R3 is 4-dimensional. Basic invariants are the 
ones of Theorem 2.2 and 


(4) Ig=E'+Ht++ct-+c0°%. 
In a way similar to J3, [4 measures a modified mod 2 number of direct 


selftangencies in a generic homotopy between two mappings. 
The proof of both theorems is given in subsection 7.1. 
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2.2 mod2 winding numbers 


An oriented circle immersed into a plane has a winding number. By 
one of many equivalent definitions, this is the number of rotations of the 
tangent vector when its point of application runs along the circle once. A 
circle immersed into a 2-sphere has a well-defined mod 2 winding number: 
to obtain a plane, we puncture the sphere at any point not on the circle 
and take the parity of the plane winding number. This parity does not 
depend on the choice of the puncture and, for a generic curve, is opposite 
to the parity of the number of its double points. 

Now take a generic mapping of a surface into R°. Consider the inverse 
image D of the set of singular points of the image. This is a collection 
of circles immersed into the source surface. 


Proposition 2.5 For generic mappings of a sphere, the mod 2 invariant 
I, +I, ts the parity we(D) of the total winding number of D. 


Proof. w2(D) is the mod 2 sum of the number of connected components 
and the number of double points of D. The parity of the number of 
double points of D is the same as the parity of the number of triple 
points of the mapping. The latter changes only across the stratum C. 
The number of connected components mod 2 of D changes only across B 
and K. Since both w2(D) and J; + J, vanish on embeddings, the claim 
follows. 


3 Immersions 


3.1 Local invariants of immersions 


Let Q; C Q be the space of C® immersions of an oriented surface to R3. 
Restriction of local invariants from Q to 2; defines local invariants of im- 
mersions. Almost always this exhausts all local invariants of immersions 
(as before, the invariants are considered modulo additive constants): 


Theorem 3.1 The space of integer local invariants of immersions of an 
oriented surface to R° is two-dimensional. Basic invariants are: 


fy2=T and I,= 8? — E+ H-. 


With the normalization I,/2(embedding) = 0, I; jo(f) is the number 
of pairs of triple points of the image of f. 
There is something extra over Zo: 
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Theorem 3.2 The space of mod 2 local invariants of immersions of a 2- 
sphere to 3-space is four-dimensional. In addition to the basic invariants 
of Theorem 3.1 there are two more: 


he aug TE ified, LO) 


Here J, is the restriction of the invariant of Theorem 2.4 and will 
appear for any oriented surface, not only for the sphere. The reason why 
we formulate the theorem for spheres only is as follows. 

Considering bifurcations in all possible generic 2-parameter families 
of immersions of a surface (as in proofs of the first three theorems) it 
is easy to show that a small loop in 9; around the set of nongeneric 
points of the discriminant intersects the stratum @ an even number of 
times. But 9; has nontrivial fundamental group. A well-defined invari- 
ant should have zero increments along all elements of 7(Q;). I do not 
know generators of this fundamental group in the case of an arbitrary 
surface. But for the 2-sphere, 7 of the space of marked immersions (i.e. 
the ones that send a distinguished ordered 2-frame of the tangent space 
to S$? at a distinguished point to a distinguished ordered 2-frame at a 
distinguished point of R*; for our needs it is enough to consider only 
marked immersions) is freely generated by a loop [15] described in [9]. 
This loop intersects @ twice. 

It would be very interesting to work out whether J, survives the 
7m(Q,)-test for surfaces of positive genus. 

Of course, there are no such difficulties with 7 for the restrictions of 
the local invariants from 2, which is contractible. 

Proofs of Theorems 3.1 and 3.2 are given in subsection 7.2. 


3.2 Sphere eversions 


Definition 3.3 An eversion of a sphere is a turning of a positive sphere 
inside out in R® by a generic regular (ie. with no pinch points) homo- 
topy. 

Eversions of surfaces are possible due to the classical result of Smale 
[15, 16]. 

Let N(S) be the number of crossings of the stratum S (the signs 
of the crossings of a cooriented stratum are respected) during a sphere 
eversion. 


Corollary 3.4 The numbers of bifurcations during a generic sphere ev- 
ersion are subject to the following relations: 


N(T)=0, N(E?)—N(B°)+N(H-) = -1, N(E')+N(H*) = 0mod 2 
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Proof. Consider a generic path in the space of mappings of S* into R? 
along which a positive sphere becomes a negative one via the birth of 
a negative bubble and death of a positive one (Fig.6). The right-hand 
sides of the first three relations are the increments of the local invariants 
of Theorems 2.2 and 2.4 along this path. 


OrG-O#0 


Figure 6: Nonregular homotopy turning a sphere inside out 


Another way to prove the Corollary, not using any particular path, 
follows from the description of an integral invariant of the next section. 


Remarks. a) There is no suprise in the restriction N(T) = 0: having 
generated triple points, we must kill them to complete an eversion. 

b) One more restriction, N(Q) = 1mod2, was proved by Max and 
Banchoff by explicit presentation of an eversion with one quadruple point 
[9]. This implies, by the way, that, during an eversion, there are at 
least two positive crossings of T to create the vertices of the tetrahedron 
vanishing on @. The basic loop in Q;, mentioned in the comments after 
Theorem 3.2, is a composition of the eversion of [9] with its ‘mirror 
image’. 


4 Integral invariant 


We introduce now an invariant of generic mappings from oriented sur- 
faces to 3-space that is very similar to the integral in Rokhlin’s complex 
orientation formula for real algebraic plane curves [12, 13, 14, 19]. This 
invariant turns out to be local, very closely related to Is. 


4.1 Degrees 


Let Imf be the image of a generic mapping of a surface 
f : M > R®. Take a point u in R? not on the image. Consider a 
small 2-sphere, with the outer coorientation, centered at u. The radial 
contraction of the image onto the sphere defines a through mapping from 
M to the sphere. We denote by deg(u) the degree of this mapping. 
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Imf subdivides the ambient 3-space into a finite number of connected 
components D. deg(u) is constant on each of them. We denote the 
corresponding value by deg(D). 

We define an integral of function deg against Euler charactestics x 
setting [19, 20]: 


Frosing 429(edex(H) =H deal D)X(D), 


where D runs through all the connected components of R* \ Imf. 

There are 8 (resp. 3) local connected components of the complement 
to the image around a triple point t (resp. a pinch point p). We set 
the degrees deg(t) and deg(p) to be the arithmetical means of the cor- 
responding 8 or 3 degrees. deg(t) is a semi-integer that also coincides 
with the arithmetical mean of degrees of any of four pairs of the ‘oppo- 
site’ local components. deg(p) is the degree of the ‘largest’ of the three 
components around p (Fig.7). 


ral} 
fer CIP 


Figure 7: A triple point of degree d — 3 and a pinch point of degree d. In 
both cases d is the degree of the distinguished component of R? \ Image 


4.2 The invariant 


We set 
166A = foo ga, ealwdex(e) — D aeal ~ 5X cea) 


where ¢ and p run through all the triple and pinch points of the image. 


Example 4.1 The value of J pona standard immersion of a surface of 


genus g (Fig.8), with the outer coorientation, is 1—g. 


Theorem 4.2 I ii is a local invariant. Up to an additive constant, 


Ip = 2s ~ le — Ip. 
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Figure 8: Standard immersion of genus 3 surface 


Thus all the three local integer invariants of Theorem 2.2 may be 
defined only in terms of geometry of the image of the mapping f. There is 
no need to choose any homotopy in 2 between the distinguished mapping 
and f. 

The assertion of Theorem 4.2 is parallel to the assertion of Viro’s 
Corollary 3.2.B in [20] for the invariant J~ of plane curves. 

The proof of Theorem 4.2 consists in the comparison of the jumps of 
I ii and the above combination of the local invariants. It will be given in 


subsection 7.3. 


Remark. The definition of [ f is similar to the formula for calculating the 
winding number w of a generic plane curve [20]: 


w => deg(D) — S~ deg(d) 


with D and d running through all the connected components of the 
complement to the curve and all the double points, deg(D) defined in 
the obvious way, and deg(d) being the arithmetical mean of the four 
numbers. 


4.3 Smoothings of images 


The intergal invariant J f has another description, in terms of a smoothed 
image. 


Lemma 4.3 There is a canonical way to smooth singularities of the im- 
age of a generic mapping of an oriented surface to 3-space. This smooth- 
ing 1s given by local pictures of Fig. 9. 


Let Im f be the smoothed i image of a generic map f. As in subsection 
4.1, for each point u of R? \ Imf, we define deg(u). 


Theorem 4.4 


1p) = faa ep 4200e)x(0) ~ deol) 
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Figure 9: Local smoothings of the image of a generic map 


Here p runs through all the pinch points of Imf. 

The assertion of Theorem 4.4 is parallel to the assertion of Viro’s 
Theorem 3.1.A in [20] for the invariant J~ of plane curves. The proof of 
Theorem 4.4 is given in subsection 7.4. 


4.4 Connected summations 


Let us choose a system of constants a(g) for the system of local invariants 
p f -++a(g) on the spaces of mappings of surfaces of genus g to R°, in order 
to get a good invariant on the union of all these spaces. As in (aileettorsthe 
notion of ” goodness” let us take additivity with respect to the connected 
summation. It turns out that we have to distinguish two types of such 
summation. The sets of the constants a(g) for the two types are distinct. 

A connected sum of two surfaces M, and Mg, separated by a plane, 
in R3 is defined as shown in Fig.10: we cut a small disc out from the 
‘exterior’ part of each of the surfaces and join the surfaces by a thin 
cylinder circlexinterval embedded in R? \ (M, U M2). If we want to 
save the orientations of the summands, there is an obvious restriction on 
My, and Mg. Coherently with the coorientations of the summands, the 
connecting cylinder must have either the outer or inner coorientation. We 
call these two options the positive and negative summations respectively. 


Theorem 4.5 The invariant I f +ag+g-—1, where a is an arbitrary 
number, is additive with respect to the positive connected summation. 
The invariant I f +6q—9+1, where B is an arbitrary number, is additive 


with respect to the negative connected summation. 
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Figure 10: Connected summation of surfaces and its two types: positive 
and negative 


The Theorem easily follows from the summations of the standard 
surfaces (Fig.8). a and are values of the invariants on the standard 
tori, with the outer and inner coorientations respectively. 


Remark. The invariants J, and J, normalized to vanish on embeddings 
are obviously additive under both connected summations. For J3 to be 
positively (resp. negatively) additive we set its value on the standard 
surface with the outer (resp. inner) coorientation to be (ag + g — 1)/2 
(resp. (@g — g + 1)/2). 

The best behaviour of I fi is under twisted summation, when we use a 
tube with two pinch points as a connecting bridge (Fig.11). Ip is additive 
with respect to such a summation. But now we need to shift J,: to make 
it twisted-additive we set I) = Ip + 2. 


Figure 11: Twisted connected summation 
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5 Linking invariant 


In (2, 3] Arnold gave an interpretation of his plane curve invariant Jt in 
terms of linking numbers of corresponding legendrian curves in different 
covers of ST*R?. Below we introduce a similar presentation for I. We 
need there some extra work since a generic mapping of a surface into 
3-space has critical points, while a generic plane curve does not. 


5.1 Legendrian lift of a generic image 


Let f be a generic mapping of an oriented surface M into oriented 1 
We are going to associate to f a subvariety LF of the contact variety 
ST*R? of cooriented tangent 2-planes of R?. i will be homeomorphic 
to M, and the image of Ly under the eran aq: ST*R? — R’ will 
be the image Imf of f. 

The part of Ly over nonpinch points of Imf, is the usual legendrian 
lift against 7: it eee to a point a the cooriented (as in subsection 
1.3) 2-plane tangent to a sheet of Imf passing through a. This divorces 
transversal selfintersections of Imf. 

Over a pinch point of Imf the legendrian lift gets a hole. Namely, 
easy computation shows: 


Lemma 5.1 Consider a Whitney umbrella W cooriented on the comple- 
ment to its pinch point p. Consider the closure of the legendrian lift of 
W \p to ST*R?. Then the boundary of the closure ts an equator of the 
2-sphere Se: The equator consists of all covectors vanishing on the image 
of the peroneal of a parametrization of the umbrella by plane. 


Here Sol is the fibre of the bundle a over a point p. 

To aie the hole in a canonical way, we need to recall the orientation 
of the fibre induced by an orientation of R® [4]. The induced orientation 
of the fibre S2 comes from the orientation of the boundary of a small 
3-disc in R. The set of outer normals of the boundary becomes a fibre 
when the radius of the ball tends to zero (vectors are identified with 
covectors by means of a metric on R®). 

Let v: R? > W be a parametrization of the umbrella by an oriented 
plane. We patch the hole on the legendrian lift of the punctured umbrella 
with a half of the fibre $2. We chose the half so that the orientation 
induced from the plane a the lift extends to the patched surface as the 
canonical orientation of the fibre. 
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Lemma 5.2 Choose a vector v in the image of the differential of v at 
the pinch point so that v is directed to the half of the umbrella with the 
outer coorientation (Fig.12). Then the patching hemi-sphere is the set 
of covectors nonpositive on v. 


Figure 12: Covectors added at a pinch point to patch the legendrian lift 


Doing the patching for all pinch points of the image of a generic 
mapping f of M, we get the subvariety LF GS ke 


5.2 Intersection with the 3-chain 


There is another way to lift Imf to ST*R?: we send a point to the 
negative of the coorientation normal (cf. [2, 3]). Let us lift a punctured 
neighbourhood of a pinch point is this negative way. The closure of the 
lift has the equator of Lemma 5.1 as the boundary. The orientation of 
the boundary equator is easily seen to be the same as for the lift in the 
previous subsection. Thus, the patches compatible with the orientations 
are also the same. We denote the new lift of the image (with the old 
patches used) by L7 (it is homeomorphic to M as well). 

Now we define the linking number of L? and Lj. 

Let us choose a direction in R? and shift L; by all possible vectors 
of this direction. We get in ST*R® a 3-chain V; with the boundary ID 
We orient it as the image of R, x M, where R, is the nonnegative axis 
oriented from 0 to oo. 

In order to bring Ly in more or less general position with respect to 
V, , we slightly shift every point of Ly in the direction of the correspond- 
ing normal vector. Namely, a point (a,n) € ST*R* moves to (a+en,n), 
for small constant € > 0 (a is a point in 3-space, n is a (co)vector applied 
at a). Let L}, be the shifted variety. 

We take the canonical orientation of ST*R* by the frame (positive 
frame of the base, positive frame of the fibre) [4]. 
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Definition 5.3 The linking invariant €~(f) is the intersection number 
of LF, and V;. 


Since the cohomology of R? is trivial, the definition is independent 
of the chosen direction. 


Example 5.4 The value of £~ on the standard genus g surface (Fig.8) 
with the outer coorientation is g — 1. For the inner coorientation it is 
1—g. 


It is easily seen that in regular homotopies £~(f) can change only at 
inverse selftangencies. In general the following holds: 


Theorem 5.5 f= —T ap lls 


A wrong choice of the patch for the lift of an umbrella would imply 
appearance of an extra term, like the sum of degrees of pinch points, in 
the right hand side of this formula. Such a sum is not a local invariant. 

A sketch of the proof of Theorem 5.5 is given in subsection 7.5. 


6 Higher dimensions 


The constructions of linking invariants in [2, 3] and the previous section 
suggest introduction of the following two linking numbers for generic 
immersed oriented closed hypersurface M in Euclidean space of arbitrary 
dimension n. These numbers count direct and inverse selftangencies in 
generic regular homotopies. 

As above, just changing 3 for n, M defines two legendrian subvari- 
eties, Lt and L~, in ST*R”. There are no patches now. Choosing a 
direction in R”, we get n-chains Vt and V~. As in subsection 5.2 we 
perturb and get Ly. 


Definition 6.1 ¢+(M) and £-(M) are indices of intersection of LE with 
V* and V~ respectively. 


There is a discriminant in the space of C® immersions of a fixed 
hypersurface into R”. The top strata correspond to tangency of asmooth 
sheet to the transversal intersection of r < n other smooth sheets (any r 
of these r +1 sheets are in general position; for r = n ‘tangency’ means 
‘passing through’) [11]. 


Lemma 6.2 The invariant €+ (resp €~) can change only across the 
strata of direct (resp. inverse) selftangency of two smooth sheets. 
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The claim is obvious from the definition. 

We can locally represent the selftangency bifurcation as moving a 
hyperplane x = t, t € R, through the graph x = —y? —...-y? + 224+ 
...+22,u+v=n-—1. Assume the frame (2, 91,..-; Yu, 21,---,2y) gives 
the orientation of the ambient space and x is the direction to produce 
the n-chains V* and V~. Changing the parameter t from negative to 
positive, with coorientations of the sheets as in Fig.13, we see: 


(1) in inverse selftangency bifurcation €~ has a jump +2; 


(2) in direct selftangency bifurcation in even-dimensional ambient space 
(i has.a, jump 2, 
(3) in direct selftangency bifurcation in odd-dimensional ambient space 


£* does not change. 


dim y =u, dimz =v 


ih - B 


jump of /* =(-1)"= C1y 


ak ~ fh 


jump of 7 =(1y ery 


Figure 13: Jumps of local values of the linking selftangency invariants 


This shows why there are two integer selftangency invariants, 
Arnold’s J* [1, 2, 3], for plane curves, and only one, @~, for surfaces 
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in 3-space. The fact of (3) reflects existence of locally noncoorientable 
direct selftangency substratum, like H*, in higher dimensions. 

As in the case of plane curve invariants, we can coorient the inverse 
selftangency stratum in the direction of decrease of ¢~ and the direct 
selftangency stratum, for R°’¢", in the direction of increase of ¢+. For 
plane curves this gives us’ J + up to additive constants [2, 3]. For ambient 
R3, the first choice gives exactly E? + H~ — E°. 


Example 6.3 On the standard (n — 1)-sphere in R”, with the outer 
coorientation, 2+ = (—1)""' and @- = —1. On the sphere with the inner 
coorientation + = £~ = (—1)""?. 

Since the 6-sphere in R” is evertible [16], we get 


Corollary 6.4 During a generic eversion of a 6-sphere, the number of 
opposite selftangencies is odd. 


Counting the signs of crossings of the discriminant, this number is 
le 


7 Proofs 


7.1 Proofs of Theorems 2.2 and 2.4 on enumera- 
tion 
of local invariants of generic maps 


Consider a function ¢ on the set of the top strata S of the discriminant. 
Let us integrate ¢ along a generic loop ¥ in {2. For this, taking zero as 
the initial sum, we follow y and, every time when it intersects a stratum 
S, add either $(S) (if the intersection is done in the direction of the 
coorientation of the stratum) or —¢(S) (if the intersection is done in the 
opposite direction). 

{(S)} is the set of jumps of a local invariant along the strata if and 
only if the integral of ¢ along any generic loop is zero (then the invariant 
is uniquely defined by {¢(S)} up to an additive constant). This gives a 
system of linear equations on the values o(S). 

Since Q is contractible, the equations should only express vanishing 
of the integrals along small loops in 2 around the set of nongeneric points 
of the discriminant (the latter has codimension 2 in Q). Such a loop is 
realized as a loop around the origin in the parameter space of a germ 
of a generic 2-parameter family of mappings from our oriented surface 
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to R3. So we will look through all such deformation and write out the 
linear system on the jumps. 

The jump along a stratum S will be denoted by s. The coorientations 
of the strata are as in Sect.1. Unknowns h* and gq’, the jumps along 
the noncoorientable strata, may be nonzero only for Zo-invariants. The 
complete list of the 2-parameter families, particular normal forms and 
bifurcation diagrams are provided by general machinery of the Theory 
of Singularities [10, 5, 6, 7], and in what follows we do not consider any 
details of the classification procedure. 


7.1.1 Uni-germs 


The following is the complete list of generic 2-parameter families of map- 
germs (R?,0) — R* [10]: 


Ap : x,y’, y(y? + 23 + Ax + p) 
By: x,y’, y(a? + y* + Ay? + p) 
Hp: az, cy + y?(y? + Ay + 1), y® + Ay 


The bifurcation diagrams for the two different coorientations of the 
sheet, along with generic members of the families, are shown in Fig.14. 
Walking counterclockwise around the origins in the parameter planes we 
read the equations: 


Bo: (1) bt—kt=0, (2) bo —k-=0 
By: Cy SS te =) Cee 0 
Ba (5) e?+k~—bt=0, (6) e®+kt—b- =0 
Hp: CO) Ge aa — 


7.1.2 Bi-germs 
Degenerate tangency of two smooth sheets 
p= and z=2rtyt+raytyp 
provides (Fig.15): 
(8) @=nA, Qe he do) = 


Interaction of a smooth sheet with an umbrella. At the pinch point, 
the smooth sheet may be nontransversal either to the tangent line to 
the handle or to the image of the differential. The latter case has two 
subcases. All this is shown in Fig.15. The families are families of parallel 
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Figure 14: Bifurcations in generic 2-parameter families of uni-germs 
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Figure 15: Bifurcations in generic 2-parameter families of bi-germs 
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translations of the umbrella. (A, 4) are the coordinates of the pinch point. 
We get 8 equations: 


(11) tte = (12) amc! =i 

(13) eet (14) Cameco = 1° 

(15) c++e=ct+e? (16) ct +&=c +e! 
ale ctact+ht+h (18) co =c +ht+h. 


7.1.3 Some 3-germs 
Lemma 7.1 (CO) 


Proof. Consider the interaction of three smooth sheets in Fig.16. A and 
uw are the coordinates of the vertex of the parabolic sheet. From the 
bifurcation diagram we get t? = t?. Changing the coorientations of the 
sheets we obtain ¢? = ¢! = 7°. 


ra 


Figure 16: Bifurcation showing that t? = ¢? 


Lemma 7.2 (19) q=0. 


Proof. Consider the interaction of an umbrella and two transversal 
smooth sheets (Fig.17, parameters \ and ps are the coordinates of the 
pinch point). Walking counterclockwise around the origin in the param- 
eter plane we read the equation: 


ee cc gr ease 0, 


Thus, by the previous Lemma, g® = 0. Changing the coorientations of 
the smooth sheets, we get g* = q” = 0. 


Remark. We have distinguished two directions at the pinch point: the 
tangent line to the handle and the image of the differential. In the 
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Figure 17: Interaction of an umbrella and two transversal smooth sheets 


proof of Lemma 7.2 we could take a different umbrella: with these two 
directions, for A = 4 = 0, not separated by the smooth sheets. Similarly, 
in the proof of Lemma 7.1 we could take a hyperbolic sheet instead of the 
elliptic one. These changes would have no influence on the final results. 

There are some more generic 2-parameter families of mappings involv- 
ing only smooth sheets: quintuple point, a sheet passing through three 
coordinate planes with tangency to one of the coordinate lines, second 
order (degenerate) tangency of a smooth sheet and the line of transversal 
intersection of two others. But, it follows from the two lemmas above 
that these families provide no further independent equations. 


Solving the system (1-20) over Z and over Z» we get the systems of 
jumps claimed in Theorems 2.2 and 2.4 respectively. 


7.2 Proofs of Theorems 3.1 and 3.2 on enumera- 
tion 
of local invariants of immersions 


Due to the comments made after the formulation of Theorem 3.2 in 
Sect.3, we have to consider only local events in generic 2-parameter fam- 
ilies of immersions. This leaves us with the twelve ‘smooth’ unknowns 
and the above equations (8-10) and (20). But now we will obtain ex- 
tra information from some bifurcations that we did not need to study 
attentively at the end of the previous subsection. 


Lemma 7.3 (21) g=q@=¢a'. 


Proof. Fig.18 shows bifurcations of a sheet u-+w = (v +»)? + pw with 
respect to the coordinate planes in the (u,v, w)-space (for \ = p = 0, 
the sheet passes through the origin being tangent to the v-axis). This 
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implies g? = g*. Changing the coorientation of, say, the curved sheet we 
get the other equality. 


a 
n ” ws 
Oo 


Figure 18: Bifurcation showing that q* = q* 


Since Q? cannot be cooriented by local means, an integer invariant 
has no jumps along the whole of the Q-stratum. Jumps of a Zo-invariant 
must be the same along all three substrata of Q. 

The bifurcation diagram of a quintuple point (five plane passing 
through one point) consists of 5 Q-lines. This introduces no new equa- 
tions. 

The bifurcation diagram of a smooth sheet u = v? + Av + p with 
respect to two sheets u? = w® (for A = 4 = 0, the first sheet has second 
order tangency with the line of intersection of the two others) is a cusp 
of T. Following the coorientations of the sheets we get either it oF 
t} = t?. Once again, nothing is new. 

Now Theorems 3.1 and 3.2 follow as solutions to the system (8-10), 
(20),(21): 


7.3. Proof of Theorem 4.2 about the integral in- 
variant 


The equality I les 213 — I, — Ip up to an additive constant is equivalent 
to the equality of jumps of both sides along all the strata. So, let us 
compare the jumps. We use the obvious 


Lemma 7.4 Crossing a sheet against its coorientation increases by 1 the 
degree of a connected component of the complement of the mage. 


Bt, Fig.19. Here d is the degree of one of the components of the 
complement to the image. It determines the degrees of the other com- 
ponents. We calculate the difference of values of Ir on two mappings 
whose images differ only inside a small ball, where we substitute the 
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right picture of Fig.19 for the left one. Due to the additivity of Euler 
characteristics, the calculations can be done locally [19, 20]: 


b* (Ip) = (4-1) +d + (d +1) - 5+) — ((d-1) $4) =1. 


Similarly, b- (I 7) =-1. 


geal Gall 
deg pj =d 


Figure 19: Distribution of degrees for B* bifurcation 


C++, Fig.20. We have: 
im ie 1 
oF (Ip) =((d+1+4+1)-—(d+ an 1))~ 5d= +1. 


Similarly, c®~ (I) = +1. 

All the calculated jumps coincide with the jumps of 21. 3—1:—1,- Note 
that jumps c%t and b+ specify a local invariant on 2 uniquely. Thus it 
only remains to prove locality of J. 

There are no more direct calculations needed. 

Indeed, assume we need to calculate a jump of I along Kt, with a 


given distribution of degrees for the components of R? \ Imf (the dis- 
tribution is completely defined by the degree of any of the components). 
We can realize such a jump within the 2-parameter A»-family (Fig.14) 
with the appropriate distribution of the degrees (Fig.21). The difference 
of the values of I on two particular mappings is determined indepen- 
dently of any path connecting the mappings in Q. So, the jumps of J f in 
the Ap-family along K+ and Bt are equal (see the bifurcation diagram 
of Ag in Fig.14). This means that the jumps of Ty and 2/3 — I, —I, along 
K* coincide. 

In the similar way, jumps of I i known by now successively determine 
the following jumps of J ih (for any possible distribution of the degrees). 


(1) from Fig.14. k-=-1, ho =2, @=2, C= —2; 
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degt=d+1/2 degp= dtl 


iy 


degt=d+1/2 degp = dtl 


Figure 20: Distribution of degrees for C*** bifurcations 


d d 
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Figure 21: Embedding of K* bifurcation into Ag-family 


252 V.V. Goryunov 


(2) romeo Se — 0.7 Oil 2 see 0 omen ee, 
(Giromnicd7 7. —¢ —¢ — 0) 


Thus, J ii is a local invariant. 


7.4 Proof of Theorem 4.4 about the smoothed 
form 
of the integral invariant 

Values of I f and I ra deg(D)x(D)- deg(p) on embeddings coincide. 


So, as in the proof of Theorem 4.2, it is sufficient to show that the jumps 
of I if and I f coincide along B* and C®*. Fig.22, that shows smoothed 


Bt and C** transformations, gives the following jumps of I f 


bt = ((d-1)-14+d-2+(d+1)-1-(d+a)- 
—((d-1)-14+d-1)=1, 


ctt = ((d—1)-1+d-04 (d+1)-24 (d+2)-1-(d4+1))- 
—((d—1)-1+d-1+(d+1)-2-d) =1, 


e+ =((d-1)-14+d-14+(d+ 1)-1+(d+ 2)-1—(d+1))— 
—((d—1)-1+d-14+(d+1)-14+(d+2)-1-—d) =-1. 
Changing the coorientations of the sheets to the opposite ones inter- 


changes the signs + and — in the expressions of the degrees. This gives 
for Tp: = Ihe = — am =) 


All the calculated jumps are the same as for J f- Thus, [ tes I f 


7.5 Sketch of the proof of Theorem 5.5 
about the linking invariant 


The invariant ¢~ certainly has zero jumps along the strata OOP IEG 
Direct computations show: 


Lemma 7.5 ee el, ba) = 


As in subsection 7.3, Lemma 7.5 defines all the other jumps of €~ via 
2-parameter bifurcations. Thus £7 is local. The values of ba er Eiaral i”, 
together with the initial conditions checked on standard embeddings of 
surfaces of genus g (subsections 4.2 and 5.2), identify the invariant - as 
—I ii +l. 
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deg p = d+1 


Figure 22: Bifurcations of smoothed images 
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Theorem on six vertices of a plane 
curve via Sturm theory 


L. Guieu, E. Mourre and V. Yu Ovsienko 


Abstract 


We discuss the theorem on the existence of six points on a convex closed 
plane curve in which the curve has a contact of order six with the osculating 
conic. (This is the “projective version” of the well known four vertices 
theorem for a curve in the Euclidean plane.) We obtain this classical fact 
as a corollary of some general Sturm-type theorems. 


1. Introduction 


The well known classical theorem states that a convex curve on the 
Euclidean plane has at least four vertices (critical points of its curvature). 
This theorem has been frequently discussed in mathematical literature (see 
[1,8]). Beautiful applications of this theorem to symplectic geometry were 
discovered by V.I. Arnol’d [1,2,3]. The relation to the Sturm theory is given 
by S. Tabachnikov [8]. His proof of the four vertices theorem is based on 
the fact that a function on S? without n first harmonics of the Fourier 
decomposition vanishes at least 2n times. 

A point on a locally convex plane curve c is called sertactic if the 
oscullating conic has a contact of order > 6 with c in this point. (Recall, 
that in a generic point the contact is of order 5 since a conic is defined 
by 5 points.) Sextactic points can be defined also as critical points of the 
affine curvature or by the fact that the projective length element of curve 
c vanishes in these points. 

Sextactic points are invariant under projective transformations. This 
kind of singular points is an analogue of vertices in projective (or affine) 
geometry. (Recall that in the Euclidean case the osculating circle has a 
contact of order > 4 with the curve in any vertex.) 

The following classical theorem can be considered as the “projective 
analogue” of the four vertices theorem. 


Theorem. [Six vertices theorem] A closed convex curve on R? has at least 


siz sextactic points. 


Corollary. The affine curvature of a conver closed curve on R? has at 


least six critical points. 


The proof can be found in [5]. 
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The main result of this paper is a series of general Sturm-type theorems 
(in the spirit of the Tabachnikov theorem). We estimate the number of zero 
points of a function on S! orthogonal to solutions of a disconjugate linear 
differential equation. This approach contains at the same time the four 
vertices theorem of Euclidean geometry and the six vertices theorem. 


2. The Sturm theorems 


Consider a linear differential equation on S!: 
Ad(z) = 6" (c) + un—1(2) $Y (cz) +++ + uo(z)4(z) =0 (1) 


Here u;(z) € C™(S*) (this means, all the potentials u; are periodic: u;(a2+ 
i) — 2, (a): 


Definition. Equation (1) is called disconjugate on S? if: 

1. Order n = 2k + 1: all the solutions are periodic: ¢(z +1) = ¢(z) 
and have at most 2k zeros (with multiplicity) on S?. 

2. Order n = 2k all the solutions are anti-periodic: ¢(x + 1) = —¢(z) 
and have at most 2k — 1 zeros (with multiplicity) on S?. 

In these cases, A is called a disconjugate operator. 


2.1. Theorem 1. Given a function f € C™(S1) orthogonal to all the 
solutions of a 2n+ 1-order disconjugate equation: 


J te ete)ae = 0 
gi 


then f has at least 2n+ 2 distinct zero points on S!. 


This is a generalization of the Tabachnikov theorem [8] stating the same 
fact for a function f(x) € C(S') without n first harmonics. Indeed, such 
a function is orthogonal to the solutions of the equation 0,(02 + 1)(82 + 
4)--- (02 +n?) =0. 


Corollary. A function f in the image of a 2n 4+ 1-order disconjugate 
operator A (f = Ag where g € C™(S") is any function) vanishes at least 
2n + 2 times on S!. 


Indeed, f is orthogonal to the solutions of the equation A*¢ = 0, where 
A” is the operator adjoint to A. It is sufficient to remark that the operator 
A* is disconjugate if A is disconjugate. 


Theorem 2. Given a function f € C®(S1) orthogonal to all the products 
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of solutions of a n-order disconjugate equation: 


[ £@t@)dn(e)ae =0 


then f has et least 2n distinct zero points on S’. 


Remark. There exist straightforward generalizations of Theorems 1 and 
2. It is sufficient to consider a function orthogonal to a product of any 3, 4 
etc. solutions of a disconjugate equation. 


2.2. Proof of Theorem 1. Consider a function f € C%(S') orthogonal 
to all the solutions of a disconjugate equation A@ = 0. 

First, observe that f has at least one zero. Indeed, there exists a 
solution ¢ positive almost everywhere on S 1 (take for example a solution 
vanishing in some point with order 2n, then the disconjugacy condition 
implies that it has no more zero points). From f,, f(z)¢(z)dx = 0 one 
concludes that function f changes its sign at least once. 

Let us prove that the number of points of S 1 in which function f has 
odd-order zeros (changes its sign) is superior to 2n. Suppose that f has 2k 
odd-order zero points £1,...,%2, on S 1 and k < n. Consider a solution ¢ 
with two properties: 

a) @ has a zero of order 2(n —k) +1 in 21, 

b) ¢ vanishes in all points 21, ...Z2k- 

The existence of such a solution is evident. In fact, there exists a 2k—1- 
dimensional space of solutions vanishing with order 2(n—k)+1 in 2. The 
subspace of this space which consists of solutions vanishing in r2 has the 
dimension > 2k — 2, etc. Now, the disconjugacy condition implies that 

a) Points 11,..-22_ are simple zeros of ¢, 

b) @ has no more zeros on he 

Finally, (replacing if necessary dé by —@) one obtains that func 
tions f(x) and ¢(z) have the same sign sequence on the segments 
Ja, Ze[, |x2, zal, - - .|z2x, £1[ which implies the contradiction: 
for f(2)b(z)da > 0. The theorem is proven. 


2.3. Proof of Theorem 2 is analogue to those of Theorem 1. Suppose 
that f has 2k odd-order zero points ©1,.-.,T2% on Si andk < n-1. 
Take any number s which is even if 7 is odd and odd if n is even, such 
that k < s < n-1. Then there exists a solution ¢, having odd order 


zero points in 71,...,2%5 and such that it has no more zero points on S? 
(see above). In the same way, there exists a solution ¢2 having odd order 
zero points in %s41,---, L2k and such that it has no more zero points on 


S1. Their product ¢1¢2 has the same sign sequence as f on the segments 
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]x1, 22,22, r3[,...]z2%,21[ which implies the contradiction: 
des f(x)¢1¢2(x)dx # 0. The theorem is proven. 


3. Affine and projective lengths; 


affine and projective curvatures 


We recall some classical definitions of affine and projective geometry 
of curves. It is very interesting to compare the notion of length in the 
Euclidean, affine and projective cases. If in the Euclidean case it measures 
in some sense the distance between the curve and a fixed point, then the 
affine and the projective lengths measure respectively: the distance between 
the curve and a straight line, and the distance between the curve and a 
conic. 


3.1. Affine length. Consider a parametrised locally conver curve c(x) = 
(ci(z), c2(x)) in R? (a curve without inflection points). For any z, vectors 
c’(x) and c"(zx) are linearly independent. Define the element of affine length 
by 


Then @ is called the affine parameter. 


3.2. Affine curvature. Vector c’”(x) is a linear combination of c/ (x) and 
e"(x): e” (x) = a(x)c’(x) + b(x)c!(x). Moreover, the affine parameter a is 
characterized by the fact that c’”"(c) is collinear to c’: 


(0) = k(a)e(2) (2) 
Function k(c) is called the affine curvature. 


Theorem 3.3. Wilczynski-Cartan construction [6], [9] (see also [7]). 
(i) A parametrised locally convex curve c(z) € RP? canonically defines a 
linear differential equation of the form: 


o'" (2) = w(x) d(x) + o(x)4(z) (3) 


(ti) Any equation (3) uniquely defines a locally convex curve oe) Cc RP? 
(modulo projective transformations of RP? Ib 
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Proof. To associate a locally convex curve with an equation (3), con- 
sider space E of solutions of (3). Let V, C E consists of solutions van- 
ishing at the moment z. One has a family of 2-dimensional subspaces 
in a 3-dimensional linear space, or in other words, a curve in RP?. It 
is locally convex (which is easy to verify). In homogeneous coordinates, 
c = (¢1(z) : 62(z) : $3(z)) where ¢1(z), 2(x), 63(x) are any linearly inde- 
pendent solutions of (3). Therefore, the equation (3) is uniquely defined by 
the corresponding curve. 


Figure 1. 


Lemma 1. The equation (3) corresponding to a closed conver curve 1s 
disconjugate. 

Proof. Consider a closed convex curve ¢ C RP? (see fig.1). Such a curve 
has at most two points of intersection with any projective line RP! c 
RP?2. In homogeneous coordinates c = (¢1(z) : $2(z) : $3(x)) where 
1(x), 62(x), ¢3(z) are solutions of the corresponding equation (3) (see Sec. 
2.3). Therefore, any solution of (3) is periodic and has at most 2 zeros on 


So 


3.4. Projective length. Rewrite (3) in more symmetric form: 
g(x) = 5 [(z)¢"(2) + («(x)$(x))'} + h(z)¢(z) (4) 


where h(x) = v(x) — «/(x)/2. Remark here that the operator Ap = 02 — 
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3(«(x)0, + O;4(x)) is antisymmetric. 


Definition [6]. The 1-form on c do = h(x)? dz is called the projective 
length element. 


Remark. The quantity h(x) transforms as a cubic differential h(x)(dzx)? 
by coordinate transformations. Therefore, the 1-form do is well defined 


(see [7]). 


The projective length shows how much the curve differs from a conic. 


Lemma 2. c¢ is a conic if and only if h = 0. 
Proof. Consider a second order equation 


v"(a) = Dye) 


Verify that the solutions of the equation Agd = 0 are given by quadratic 
polynomials in its solutions. In particular, ¢, = wz, be = vite, ¢3 = we 
(where 1, 2 are linearly independent) is a basis of solutions. TODS og = 
digs and the curve c = (¢1(z) : do(z) : b3(x)) is a conic. 


3.5. Projective curvature. Let us suppose that do 4 0 and so a defines 
a local parameter on c. Then, the function «(c)/4 is called the projective 
curvature of the curve c(z). 


3.6. An affine curve as a projective curve. Consider a standard 
embedding R? <> RP? preserving the projective structure on R2 (see 


fig.2). 


Figure 2. 


An affine locally convex curve c C R2 is embedded to RP? as a pro- 
jective locally convex curve. To define its projective length and projective 
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curvature, represent the equation (2) in the form (4): 


il 1 
e"(0) = slk(a)e'(o) + (Ra )e(o))'] — sk’ (o)e(o) 
Therefore, the projective length of c can be defined by the relation: 
Ih 
he) = mig (co). 


On the other hand, any projective curve can be considered (locally) as an 
affine curve. The equation (3) reduces to the form (2) by a changing of the 
parameter. 


4. Sextactic points 


Definition. A point of a locally convex curve c C RP? is called seztactic 
if there exists a conic in RP? which has a contact of order > 6 with ¢ in 
this point. 


4.1. Critical points of the projective length. The notion of a sextactic 
point can be expressed in terms of the curvature (in affine case) and in terms 
of the length element (in projective case). 


Proposition 1. A point of a locally conver affine curve c C R?2 is sextactic 
if and only if it is a critical point of the affine curvature. 


Corollary. A point of a locally conver curve c C RP? is seztactic if and 
only if the projective length element do vanishes at this point. 


Remark here that this statement is just an infinitesimal version of 
Lemma 2. 


Figure 3. 
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Proof of the proposition. Consider a locally convex curve c C RP”. 
Take the affine parameter on c, then the coordinates of c satisfies the equa- 
tion (2). In the neighborhood of point co = c(0) curve c is given by the 
Taylor series: 


2 3 4 5 
-_ / Ge a a a OF iV ne Vv 
ca) =ocot+ aio as G00 1 ry + yop +... 
From (2) one has: 
cl"! = ke’ 
a k'c! + ke” 
cY — (k" + k?)c! + 2k’ cl! 
and finally 
oe ot oF Ge oe 
= ko +ko— +... Jeg + (— + ko— + ki — 4...) 
c(o) = (o + 0G thoog + ep trae a 054 + 0790 + ) 


Fix coordinates (x,y) on RP? generated respectively by vectors c’(0) and 
c’(0) (see fig.3). 
Consider the following conic: 


ee 2y + koy” 


Satisfy the coordinates of c() to function F(z, y) = x? — 2y + koy?. One 
obtains: 


F(2(0),y(o)) = hose + 


Thus, the conic has a contact of order 5 with c and so this is the osculating 
conic to c. If kg = 0, then the order of contact is 6. The proposition is 
proven. 


4.2. Proof of the six vertices theorem. Let us show how the Six 
vertices theorem follows from Theorem 1. 


Lemma 3. The parameter h(x) in equation (3) satisfies the following 
condition: 


/ 1(z)$o(x)h(x)dx = 0 
gl 


where $1(x),$2(x) are any two solutions of (3). 
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Proof. Let (x) be a solution of (3), then dh = Ap¢d. Lemma 3 follows 
now from the fact that Ao is antisymmetric. Indeed, 


Js: 1(2)$2(x)h(x)dz = i , $1(x)Ange(x) dz = 
~ Jn Ao($1(x))¢2(x)dz = — gi $1(z)b2(x)h(xz) dr = 0 


The six vertices theorem follows now from Theorem 2 and Lemma 1. 
In fact, the function h(z) is orthogonal to all the products of solutions of a 
disconjugate equation of order 3. Thus, it has at least 6 distinct zero points 
on S$! (Theorem 2). Sextactic points of a locally convex curve c C RP? 
coincide with zero points of h (Proposition 1). One obtains, that a closed 
convex curve on RP? has at least 6 distinct sextactic points. The theorem 
is proven. 


4.3. Geometrical properties of sextactic points. Let us give here two 
geometrical descriptions of sextactic points. 


A. Any curve c in general position has almost everywhere a contact of 
order 5 with its osculating conic. Nondegenerate sextactic points can be 
characterized by the fact that c does not cross its osculating conic in such 
noints (see fig.4). 


a = 


fig. 4 (a) fig. 4 (b) 


generic point sextactic point 


Figure 4. 


B. Dual curves. Let c, and cg be locally convex curves, take any two 
points p; € cy and po € cg. Then, there exists a projective transformation 
Q € PGL(3,R) such that Qp2 = pi and the curve Qc has a contact of 
order > 5 with c, in p;. Let @ be a projectively dual curve to the curve c.We 
show that c has a contact of order > 5 with ¢ in sextactic points. 


Lemma 4. A point p of a locally conver curve c C RP? is seztactic if and 
only if c there exists a projective tsomorphism I: RP™ 5 RP? such that 
c has a contact of order > 6 with I(¢) in p. 

Proof. Let C be the osculating conic of a locally conver curve c in a 
point p. Then the dual conic C € RP is the osculating conic of €. Take 
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an isomorphism I : RP?* — RP? which maps C to C and the point of 
contact of C with é to the point of contact of C with c. 


After completion of this paper we received the preprint [4] containing 
the proof of Theorem 1 and its applications to the theory of space curves. 
We also discovered unpublished results of A. Viro who gave another proof 
of the six vertices theorem using the Sturm-Tabachnikov approach. 


Acknowledgments. It is a pleasure to acknowledge fruitful discussions with 
V.I. Arnol’d and 8S. Tabachnikov. 
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The Arf-—invariant and 
the Arnold invariants of plane curves 


S.M.Gusein-Zade, S.M.Natanzon 


In [A] V.I.Arnold considered closed generic plane curves, i.e. immer- 
sions S! + R?, the images of which have no singularities except simple 
(double) self-intersections. In a generic one-parameter family of immer- 
sions three types of modifications (“perestroykas”) of generic curves can 
be met. They correspond to three natural strata in the set of non-generic 
immersions (the discriminant). These strata consist of immersions with a 
direct self-tangency (J+), with an inverse self-tangency (J~), and with a 
triple crossing (St) respectively. All three of them are coorientable. An 
invariant of generic plane curves is an invariant of the first order (in the 
sense of Vassiliev) if its change under a modification of crossing a stratum 
(in the positive direction) depends only on the stratum, but not on a (non- 
singular) point of its crossing. For closed plane curves V.1.Arnold defined 
basic invariants of the first order J+, J~, and St corresponding to the de- 
scribed strata. The invariants J+, J~, and St can be defined for so-called 
“long” curves (that is for curves going from the infinity to the infinity) as 
well. 

The method of real morsifications describes the intersection form 
(and a Dynkin diagram) of an isolated critical point of a function germ 
f : (C?,0) — (C,0) in terms of a real plane curve with only simple (dou- 
ble) self-intersections — the zero-level set of an appropriate deformation of 
a real singularity from the same stratum {y = const}. (Generally speaking 
this curve is determined by the germ f not in the unique way.) If the germ 
f is irreducible then its intersection form is non-degenerate, the correspond- 
ing plane curve consists of one component (i.e. is the image of a generic 
immersion R — R?). In this case the monodromy group over the field 
Z> is determined by the Milnor number p = p(f ) and the Arfinvariant, 
which is an element of Z2 ((J]). The Arf-invariant (as well as the intersec- 
tion form) can be defined for any long curve (in such a way that for a curve 
associated to a singularity of two variables in accordance with the method 
of real morsifications it coincides with the Arf-invariant of the singularity). 

The object of the paper is to describe a relation between the Arf 
invariant and the Arnold invariants of plane curves. 

Theorem . The Arf-invariant of (long) curves is an invariant of the first 


order, 


Arf = J~/2 mod 2. 


268 S.M. Gusein-Zade, S.M.Natanzon 


This statement seems to be the first known relation between a “con- 
ventional” invariant of a plane curve singularity (the monodromy group 
over the field Zz here) and the basic Arnold invariants of a corresponding 
(according to the method of real morsifications) generic real plane curve. 


Remark. The invariants J+ and J~ can be defined for real irreducible 
germ functions f : (R?,0) — (R,0) (the invariant St can not be defined 
for them). As a corollary of the Theorem we have the fact that for real 
irreducible singularities of functions of two variables the Arf-invariant has 
an integer lifting (that is J~ /2). 


1. Basic invariants of long curves. 


Definition. A long curve L (or for short simply a curve) is an immersion 
L : R* — R? of the line R? into the plane R?, which is generic (i.e. 
its image has no singularities except simple double self-intersections) and 
which coincides with the map x — (z,0) outside a bounded segment. 


Two curves are considered as identical if they are topologically equiv- 
alent, i.e. if one can be transformed into the other by a diffeomorphism of 
the plane, which coincides with the identy outside a bounded region. On 
the set of curves there are two natural involutions. The involution ©, is 
induced by the reflection (z,y) — (x,—y) of the plane R? with respect to 
the z-axis. The involution X2 is induced by the reflection (x, y) > (—2, y) 
of the plane R? with respect to the y-axis (with the simultaneous reflection 
x — (—2) of the line R*). On the set of curves there can be defined a sum 
operation. The sum of two curves is constructed as it is shown in F ge. 
This operation is well-defined, associative, but non-commutative. 


SSS ty = SS 


Figure 1. 


The basic invariants J+, J~, and St of (long) curves can be defined in 
the same way as for closed curves in [A]. The only difference is the following. 
The space of all immersions R* — R? is the union of infinitely many 
contractable components. Each component consists of immersions with the 
fixed index (rotation number). An invariant of the first order is determined 
by its jumps for the modifications of crossing the (defined) strata and its 
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values for representatives from all the components. As these representatives 
it is natural to use curves L; from Fig.2 (the index of the curve L; is equal 
to i). The table of jumps for the invariants J+, J~, and St can be found 
in [A]. Let 1(L) = (J*(L), J-(L), St(L)). Then the jump AJ(L) is equal 
to (2,0,0) for the stratum J* of direct tangencies, AI(L) = (0,—2,0) for 
the stratum J~ of inverse.tangencies, AJ(L) = (0,0,1) for the stratum St 
of triple points. This table defines the invariants up to additive constants 
depending on the index. The values of these invariants for the curves L; 
are determined by the following statement. 


SQ ee 


L.1 Io ly In 


Figure 2. 


Lemma 1. There is a unique way to define the invariants J+, J~, and St 
(i.e. their values for the curves L;) so that they are invariant with respect 
to the involutions D1 and D2 and additive with respect to the sum operation. 
In this case I(L1) = (—1, —2, 1/2). 


Remark. It is sufficient to require only the additiveness and the invari- 
ance with respect to the involution &;. The invariance with respect to the 
involution 2 takes place automatically. 


Proof. It is clear that I(L;) = |i|- (11). We have I(L, + L-1) = 21(L). 
The index of the curve L, + L_j is equal to zero and hence this curve can 
be deformed (in the class of immersions) into the trivial curve Lo (Fig.3). 
In Fig.3 on each stage of the deformation except the last one we have 
exactly one generic modification (of types J +, St, J~ and J~ respectively). 
Therefore [(L; + L_;) can be found as sum of jumps for crossings of the 
discriminant. It is only necessary to keep track of signs of the modifications 
which are met in the course of the deformation). 
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AL 


Fig.3. 


Remarks. 1. The values of J~ and of J~ are alwais integers (moreover 
J~ is always even). However the value of St can be rational. It can have 
(and if the index of the curve is odd it actually has) 2 as the denomina- 
tor. Therefore it would be somewhat more convenient to define the jump 
AI(L) to be equal to (0,0, 2) for the stratum of triple points. However it 
is reasonable to follow the initial definition of V.Arnold. 

2. ‘To a (long) curve L there correspond two closed curves Tee and ie 
They can be obtained from L by connecting its “ends” in the upper and 
in the lower half-plane respectively (Fig.4). It is not difficult to see that 
I(L) = (I(L+) + I(L_))/2. 


oe 


‘4 
Fig.4. 


3. The invariance with respect to the involutions di, and Ny implies that 
the basic invariants J+, J~, and St are well-defined not only for generic 
immersions R? — R? standard at the infinity but for any proper generic 
immersion R1 — R? with finite number of self-intersection points. Thus 
they are defined for a real plane curve corresponding to an irreducible germ 
f : (C?,0) — (C, 0) in accordance with the method of real morsifications. 


2. The Arf-invariant of a long curve. 
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The method of real morsifications describes two bilinear forms on an 
(integer) lattice associated to a plane curve (a symmetric and a skew- 
symmetric ones). The general definition can be found eg. in [AGV] 
(pp.115-116). These two forms coincide over the field Zz. Only this case 
will be used here. 

Let L be a plane curve (i.e. a generic immersion R! — R? standard 
at the infinity) with self-intersection points p; (i = 1,2,...,n). Let U; be 
the bounded components of the complement to the curve LD (actually there 
are just n of them too). 


Definition. The Milnor lattice (over the field Z2) associated with the curve 
L is the vector space H over the field Z2 generated by (formal) elements 
6(!) and ae corresponding to the self-intersection points p; of the curve L 


and to the bounded components U; of its complement. The elements 6() 
and Sa are called the basic vanishing cycles. 


Definition. The intersection form associated with the curve L is the bi- 
linear form (-,-) on the Milnor lattice H (with the values in Z2) defined by 
the following list of the scalar products (the intersection numbers) of the 
basic vanishing cycles: 

1) the scalar square of each element is equal to zero, Le. the form is even; 
2) the scalar product (6°, 60 )) of two basic vanishing cycles with the upper 
index (1) is equal to zero; 

3) (60), Se) is equal to the number of verticies of the component U; which 
coincide with the self-intersection point p; (this number can be equal to 0, 
1, or 2; in the last case the intersection number is equal to 0 € Zo again); 

4) the scalar product Ce 5) of two basic vanishing cycles with the upper 
index (0) is equal to the number of common sides of the corresponding 
components U; and Uj of the complement to the curve L. 

The point 2) means that the subspace, generated by the cycles {6}, 
is an isotropic subspace of the Milnor lattice H, i.e. the restriction of the 
intersection form to it is equal to zero. In fact it is a maximal isotropic 
subspace: see Lemma 2 below. 


Definition. The Picard-Lefschetz transformation corresponding to an ele- 
ment 6 € H is the linear map h: H — H defined by the formula 


h(a) = a+ (a,6)-6 (a € A). 


Definition. The monodromy group of the curve L is the subgroup of the 
group Aut(H) of the automorphisms of the Milnor lattice H generated by 
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the Picard-Lefschetz transformations np?) and be corresponding to the 


basic vanishing cycles 6) and ee 


Definition. A vanishing cycle is an element of the Milnor lattice H which 
can be obtained from a basic vanishing cycle by the action of a transfor- 
mation from the monodromy group. 


Remark. We consider the basis {6 6m of the Milnor lattice H as a 
weakly distinguished one (see [AGV]). Actually it can be considered as a 
distinguished one. However it requires a suitable ordering of its elements. 
The description can be found in [AGV] again. 


Lemma 2. The intersection form of a (long) curve is non-degenerate. 


Proof. Let C1 be the complexification of R? and let M be a real surface 
(or rather a complex curve), obtained from C? by identification of the 
preimages of the self-intersection points (which lie in R? c C1). Let M be 
a smoothing of the curve M (as a real surface it can be obtained from M 
by cutting out neighbourhoods of the double points and replacing them by 
cylinders). It is possible to show (in fact it has been done in [GZ]) that the 
intersection form corresponding to the curve L coincides with the (usual) 
intersection form in the homology group Hy (M , 22) in a certain basis. The 
last intersection form is non-degenerate (because the surface M has ‘ 
component at the infinity”). 


‘one 


In order to define an Arf-invariant one needs to have a so-called arf- 
function (or a quadratic function), i.e. a function a: H — Z», with the 


property 
a(x + y) = a(x) + a(y) + (2, y). 


Such a function is determined by its values for the basic elements. 


Definition. The arffunction of the curve L is defined to be equal to 1 for 
all the basic vanishing cycles. 


It isn’t difficult to see that the value a(6) of the arf-function of the 
curve L is equal to 1 for any vanishing cycle 6. Since the intersection form 
is non-degenerate there exists a standard basis {u;,v;} (i = 1,2,...,n) of 
the lattice H, i.e. a basis for which (i) = G0) = 0, (es) = oi; 
(the Kronecker symbol). 


Definition. The Arfinvariant Arf(L) of the curve L is equal to 


>> a(u;)a(r;). 


7 
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The Arfinvariant is well-defined, i.e. it doesn’t depend on the choice of 
a standard basis of the lattice H. The monodromy group (over the field Z2) 
of an isolated critical point of an irreducible function germ f : (C?,0) 
(C, 0) is determined by its Milnor number and the Arf-invariant ((J]). This 
Arf-invariant coincides with the Arfinvariant of a curve L corresponding 
to the germ f in accordance with the method of real morsifications. 


3. The relation between the Arf-invariant and the basic invariants. 


In this section we prove the Theorem formulated in the Introduction. 
It isn’t difficult to see that the equality from it takes place for the represen- 
tatives L; (Fig.2) of curves with a fixed index (the Dynkin diagram of the 
curve L; is the disjoint union of |z| copies of the diagram Az). Therefore 
in order to prove it for the general case it is necessary to show that the 
Arf-invariant has the jumps equal to 0 for the stratum St of triple points 
and for the stratum Jt of direct tangecies and the jump equal to 1 for the 
stratum J~ of inverse tangecies. For the stratum St it follows from the 
fact that the action of a triple point modification on the Dynkin diagram 
of a plane curve is equivalent to a change of the distinguished basis in the 
Milnor lattice ([AGV], pp.120-121). In this sense the arffunctions (and 
hence the Arf-invariants) of the curves before and after the modification 
coincide with each other. This fact is local. It means that its proof consists 
of an analysis of the behaviour only of the vanishing cycles corresponding 
to the self-intersection points of the curve and to the bounded components 
of its complement adjacent to the point of the modification. For the strata 
J+ and J~ it can not be so. The change of the Dynkin diagram of a curve 
for a tangency modification (locally) is one and the same for the direct tan- 
gency and for the inverse one (see Fig.7 below). It is possible to distinguish 
between these cases only globally. However the changes of the Arfinvariant 
for these two cases are different. 

In order to prove the Theorem we shall use two lemmas. 

The intersection form associated with a curve LD is non-degenerate. 
Therefore to each basis of the Milnor lattice H of the curve L there 
corresponds the dual basis. The dual to the basis (Ge oe } is the ba- 
sis {6{*, 6*} such that C6. tO 6") = 0, (6 60") = 
(6, 60") = Syyr (kk! = 1,2,...,7). Let Uj be the bounded component 
of the complement to the curve L corresponding to the basic vanishing cycle 
oe and let p; be a self-intersection point of L. The point p; cuts the curve 
L into three parts: two curves going from the infinity to the point p; and 
the loop 2; which goes from p; to itself. This loop is smooth everywhere 


274 S.M.Gusein-Zade, S.M.Natanzon 


except at the point p; itself, where it has a corner. Let ti —(0--U,)| be 
the index of the loop 0; with respect to the region U; (it is equal to the 
linking number of the loop Q; with a point from U;). 


Lemma 3. The element ie corresponding in the dual basis to the ba- 
sic vanishing cycle a is equal to the sum s; = x fi. 6) of the cycles 


corresponding to the self-intersection points of ie ae L. 


Proof. It is clear that CP = 0 for each basic vanishing cy- 
cle 6 ) corresponding to a self-intersection point. Hence it is suffi- 
cient to check that (s;, 60) = 6;,~. Let as consider the loops 2; as 
slates of the homology group Hi(ImL;Z2). We have (s;,60) = 


cy £(04,U;)60? 6) = Le, 6) .£(94,U;) = (0, 6)0Q;,U;) = 


u ys (5, 6 )0:, U; te In the last sum the Bae (6, 50) can be 
i:p;€OU, 
equal to 0 if the self-intersection point p; of the curve L is on the boudary 


OU;, of the component U;, twice. The desired fact follows from the formula 


Ge = ae 
i:pi€ OU, 


in the homology group H,(Im L; Z2); OU, is the element of the homology 
group represented by ine boundary of the component U;,. This formula 
gives the equality (s,, 6 )= = 6;, because £(0U;,, U;) = 6,;, i.e. it is equal to 
1 if k = 7 and is equal to 0 othewise. The cycle S= >> (6 6 oa; is 
UDi 
equal to the sum of smooth segments of the curve DL tne ae coefficients. 
Let us pass along the curve L from —oo to +oo and keep track of the 
coefficients. The coefficient is equal to zero for the first (non-bounded) 
segment of the curve L. Suppose that we pass a self-intersection point p;, 
of the curve L (say from the segment a to the segment b). If this point is not 
on the boundary of the region U;, then all the loops Q; with (6, 50) 0) 
which include the segment a include the segment b as well. Hence the 
coefficients at a and b in S are equal to each other. If the point p;, is 
on the boundary of the region U;, twice (and hence the loop Q;, does not 
participate in the sum S), then we have just the same situation and the 
corresponding coefficients are equal to each other again. At last if the point 
Pi, is on the boundary of the region U; once, then , 6) = = 1 and in 
the sum S only the loop 2;, goes along one of the segments a or b and does 
not go along the other one. Hence the coefficients at a and b in S differ by 
1. This proves the lemma. 
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The self-intersection points cut the curve L (or rather Im) into 
smooth segments (two of which are unbounded). The preimages of the 
self-intersection points cut the line R! into corresponding segments. Let 
us define the sign sgn(a) € Zo of a segment a of the curve L as the number 
of the preimages of the self-intersection points between the corresponding 
segment of the line R? and the minus infinity (or equivalently between the 
segment and the plus infinity). For a component U of the complement to 
the curve L (bounded or not) let us take a (smooth) path in the plane R? 
which goes from a point of the component to the infinity in the (say) upper 
half-plane (e.g. which coincides with the y-axis outside a bounded region) 
and which is transversal to the curve L (in particular it does not go through 
the self-intersection points of L). Let us define the sign sgn(U) € 22 of 
the component U to be equal to the sum of signs of all the segments of 
the curve L intersecting the path. It is not difficult to see that the sign 
sgn(U) is well-defined, i.e. it does not depend on the choice of the path 
from the component to the infinity. We shall call a segment of the curve L 
or a component of its complement even or odd if its sign is even or odd (i.e. 
if it is equal to 0 € Z or tol € Ze respectively). It is possible to say that 
the sign of a component of the complement to the curve L is equal to the 
number of odd segments between the component and the infinity. The both 
unbounded segments of the curve L and the both unbounded components 
of its complement are even. 

Suppose that U;, and Uj, are components of the complement to the 
curve L (possibly unbounded and possibly coinciding with each other) such 
that there exists a (smooth) path from a point of U;, to a point of U;, which 
intersects (transversally) the curve L in two points (say on the segments a 
and b, which also can coincide with each other). If at the intersection points 
the velocity vectors of the curve L point to one and the same side of the 
path, then the components U;, and U;, have different signs (Fig.5, left). If 
they point to different sides of the path, then the components U;, and U;; 
have equal signs (Fig.5, right). This follows from the fact that the number 
of (the preimages of) the self-intersection points between the segments a 
and b is odd in the first case and is even in the second one. Hence in the 
first case one of the segments is even and the other one is odd. So the 
signs of the components are different. In the second case either both of the 
segments are even or both of them are odd. So the signs of the components 
are equal to each other. 
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Fig.5. 


nm : 
Lemma 4. The sum >> £, of all the coefficients in the expression for the 
i=1 


dual vanishing cycle ae corresponding to the bounded component U; of the 
complement to the curve L is equal to the sign sgn(U;) of the component. 


Fig.6. 


Proof. We shall use the method of (mathematical) induction. The fact is 
trivial for the curve without self-intersection points (that is for the curve 
Lo from Fig.2). Let us suppose that the statement has been proved for 
all curves with less than n self-intersection points and let L be a curve 
with n self-intersection points. Let us take the first self-intersection point 
of the curve LD, that is the end point of the left unbounded segment of it. 
Without loss of generality we can suppose that this is the self-intersection 
point p,. One of the two natural smoothings of this self-intersection point 
gives a long curve (say L) with (n — 1) self-intersection points (the other 
one gives a union of a long curve and a closed curve). See Fig.6; inside the 
regions bounded by the dotted lines the curve is arbitrary (and one and the 
same). Let the component of the complement to the curve L marked by * 
in Fig.6 be U;,. This is the component which does not become connected 
with an unbounded component of the complement after the smoothing. 
However it is possible that it is unbounded initially; in this case we suppose 
the corresponding dual vanishing cycle Coy to be equal to 0. To each 
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segment of the curve L or to each component of its complement there 
corresponds a segment or a component of the complement of the curve 
L. The sign of a segment a of the curve L coincides with (respectively 
is different from) the sign of the corresponding segment of the curve L if 
and only if the segment a does not lie (respectively lies) on the loop Qp 
corresponding to the self-intersection point pp. It implies that the sign of a 
component U; of the complement to the curve L coincides with (respectively 
is different from) the sign of the corresponding component for the curve ib 
if and only if €(Qn,U;) = 0 (respectively (Qn, U;) = 1). Let 9; be the loop 
corresponding to the self-intersection point p; of the curve ib, 7 = £23; v,) 
(i = 2,...,n). If the loop 9; corresponding to the self-intersection point 
p; of the curve L passes the self-intersection point pn, (the point of the 
smoothing) then (in the natural sense) Q; =2;+2,. Othewise Q; = 2. 
Let J = {i: 1 <i < (n-—1), the loop 9); passes the self-intersection point 


pn of the curve L}, J = JU{n}. The number #J of the elements of the set 
ie Pas Bal 

J is equal to the sum J*> ¢(9;,U;,) = >> @°. By the induction hypothesis 
i=1 1 


it is equal (mod 2) to the sign of the region U;, which is equal to 0 (see the 
paragraph just before Lemma 4). Therefore the number #J of the elements 
of J is odd. A : 
We have 32 # = 2 €(9:,U;) = ¥ (Mi, U;)+ Dn, Uy) = At 
i=1 i=l i= ied i=l 
(Qn, U;). Therefore the sum >? é2 coincides with (respectively is different 
i=1 


n-1_. 
from) the sum }> @ if and only if £(Q,,U;) = 0 (respectively €(Qn, Up = 
i=l 


1). It means that this sum changes just in the same way as the sign sgn(U;) 
of the component U;. This proves the lemma. 


Corollary 1. The value a6.) of the arf—function a for the dual vanish- 
ing cycle ee is equal to the sign sgn(U;) of the corresponding component. 


It follows from the fact that the restriction of the intersection form to 
the subspace, generated by the cycles oy is equal to zero and hence the 
arf-function a is linear on it. 


Corollary 2. Suppose we have the situation described in the paragraph 
preceding Lemma 4. Let cl and i be the dual vanishing cycles corre- 
sponding to the components U;, and U;,. If one (or both ) of the components 
is unbounded we shall suppose the corresponding dual vanishing cycle (cy- 
cles) to be equal to 0. Then o(50* aa Ei) is equal to 1 in the first case 
(if the velocity vectors point to one and the same side of the path) and is 
equal to 0 in the second one (if the velocity vectors point to different sides 
of the path). 
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Proof of the Theorem. We have to consider a (direct or inverse) tangency 
modification and to determine the corresponding jump of the Arf-invariant. 
In a general case such a modification leads to the following change of the 
Dynkin diagram of the curve (see Fig.7; the thin lines show the curves 
and the thick ones show the edges of the corresponding Dynkin diagrams; 
only the indicies of the corresponding basic vanishing cycles are indicated). 
Before the modification (Fig.7, left) we have cycles 6,, 6;, and 6_ (and 
some other cycles which do not really participate in the modification). The 
words “In a general case” above mean that in particular cases some of these 
cycles can be absent (the corresponding components can be unbounded) or 
can coincide with each other (for the cycles 6; and 6_). The necessary 
changes of the proof are clear (we shall indicate them in the course of the 
proof). After the modification (Fig.7, right) we have cycles 6), 5, 63, 54, 
6s, 64, and é_. For a basic vanishing cycle 6 outside the picture (that 
is which does not really participate in the modification) we have: 1) the 
intersection numbers (6;,6) and (6_,6) are the same as those before the 
modification (it is the reason to denote the cycles 6, and 6_ before and 
after the modification in the same way); 2) (6;,6) = 0 for i = 2,3,4; 3) 
(6,,5) = (61,5) + (55,6) (if the cycle 6,, existed before the modification, 
ie. if the corresponding component of the complement was bounded). As 
earlier let us suppose the corresponding dual vanishing cycle 6% or/and 6* 
to be equal to zero if one (or both) of the corresponding components is 
unbounded. 


Fig.7. 


Let us make the following change of the basis:6; 6 =6 +6, 4+ 6, 
1 <i < 4 (all the other basic cycles are unchanged; generally speaking 
the new basis is not a weakly distinguished one). This is a change of the 
basis indeed because of the following. It is not difficult to see that the 
coefficients at 62 and 64 in Sime (in the Milnor lattice H of the curve after 
the modification) may be (and in fact are) different only for the component 
of the complement corresponding to the cycle 63. In particular they are 
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equal to each other in 64 +6*. In the Dynkin diagram this change cuts all 
the connections (edges) between the cycles 6; with 1 <i < 4 and the cycles 
6, and 6_. From Corollary 2 (after Lemma 4) it follows that for 1 <i <4 
a(6,) = 1 if the modification is of the type J~ (ie. an inverse tangency 
modification) and a(é/) = 0 if the modification is of the type Jt (ie. a 
direct tangency modification). This is the crucial point of the proof. 

The change of the basis of the form 65 ++ 6, = 65 +6; + 63 cuts the con- 
nection between the cycle 65 and the cycle 64. It changes the connections 
between the cycle 6; and the cycles outside the picture. The intersection 
number (65,6) is equal to (65,6) + (61,6) and hence it just coincides with 
the intersection number (6.,6) (before the modification) for each basic van- 
ishing cycle 6 outside the picture. This takes place in the case if the cycle 
6, existed before the modification. If it is not the case, the cycle 465 does 
not exist after the modification as well and we simply omit this change of 
the basis. This change does not influence the corresponding values of the 
arf-function. 

At last if we add the cycle oo + 64, to each basic cycle 6 outside the 
picture with (6{,6) = 1, we do not change the values of the arf-function 
and cut all the conections between the part of the diagram, corresponding 
to the cicles 6; with 1 <7 < 4, and the remaining part of the diagram. This 
splits the Dynkin diagram into two parts (and the Arf-invariant into two 
summands). The first part (which includes the cycles 64, 6_, 6, and all the 
basic vanishing cycles outside the picture) coincides with the Dynkin dia- 
gram of the curve before the modifications. The values of the arf-function 
for all the basic cycles in it are equal to 1 and therefore the Arf-invariant 
of it is equal to the Arf-invariant of the curve before the modification. The 
second part of the diagram coincides with the Dynkin diagram A,. The 
values of the arf-function for all the basic cycles in it are equal to 1 if the 
considered modification is of the type J~ and are equal to 0 if the consid- 
ered modification is of the type J+. The Arf-invariant of this part of the 
diagram is equal to 1 in the first case and is equal to 0 in the second. This 
proves the Theorem. 
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Produit cyclique d’espaces 
et opérations de Steenrod 


Maz Karoubi 


Dans cet article nous poursuivons notre étude des opérations de Steenrod 
commencée dans [3], en utilisant le produit cyclique des espaces et, indirecte- 
ment, leur produit symétrique infini par Dold et Thom [1]. Afin de faciliter sa 
lecture, nous commencerons par rappeler quelques définitions fondamentales. 

Soit X un ensemble simplicial pointé, de point base +, et soit R un anneau 
commutatif. Nous désignerons par L(X;R) (ou simplement L(X) s'il n’y a 
pas de risque de confusion) le R-module simplicial libre de base X avec la 
relation * = 0. Il est bien connu que les groupes d’homotopie de L(X) sont 
(presque par définition) les groupes d’homologie réduits de X & coefficients 
dans R. En particulier, si X est un modéle simplicial de la sphére S” (par 
exemple l’ensemble simplicial des applications continues des A, dans ae 
0,1,2,...), L(X) est un modéle de l’espace d’Eilenberg—-Mac Lane Ken) : 
tous ses groupes d’homotopie sont nuls sauf le ni&™e qui est isomorphe & R. 

Si Y est un deuxiéme ensemble simplicial connexe pointé, un accouplement 
R-bilinéaire 


L(X)A L(Y) — LX AY) 


est défini en associant & x = }); Aixi et y = D0; Hjyz Vexpression suivante® : 


xzUuy = es A Y;3- 
i,j 
Pour X = $% et Y = S” par exemple, il permet de définir une application 
K(R, 4) A K(R,7) ar K(R,q+1r). 
Elle induit le cup-produit usuel en cohomologie 
H9(X;R) x H™(X;R) — H*"(X; R) 


car H™(X; R) = [X, K(R,m)] en général. 
Un cas particulier important de cup-produit est la puissance p®™° d’un 
élément x = )>, Aix; de L(X) : elle s’écrit 


gP? = De Ni Aig + Aip Lig A Vig N--* NZ, 


41 ,424-..;8p 


1 ot A, et wy E Ret EX, yj EY. 
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dans L(X”%?) = L(X AX A--- AX) (p facteurs X). 

Supposons maintenant que p soit un nombre premier. Le p-produit cy- 
clique normalisé de X est le quotient de X”%? par l’action du groupe cy- 
clique C, permutant circulairement les facteurs, la diagonale de X AP étant 
identifiée au point base. I est noté CP}(X). Sig: X’? —+ CP}(X) désigne 
l’application canonique, il est clair que y(z?) = y((>_; A:2i)?) est la somme de 
p copies de g(x), ot l’application simpliciale 


gp: L(x) — L(CP}(X)) 
est définie par la formule suivante 


p(x) = (>> Axi) = ‘s Ai Nig? Ad, PEI, A Bin A>? Nz), 


(51 fpgscctia) 


(21, %2,-.-,%p,) désignant la classe de la suite (¢1,%2,...,7)) modulo l’action du 
groupe cyclique C,. En suivant la terminologie de Steenrod [5], qui est tout a 
fait adaptée a notre contexte, nous appellerons g(x) la p-puissance réduite 
de x = 50, A;2;. Cette décomposition de y(z?) est analogue simplicial de la 
propriété algébrique bien connue: 


Oo ai)? — ie divisible par p, 


a 


ou p est un nombre premier, les a; étant des variables commutant circulaire- 
ment (dans un anneau non nécessairement commutatif). 
Si p = 2 par exemple, on a 


t ol) 


Le p-produit cyclique (non normalisé) CP,(X) est le quotient de X”? 
par l’action du groupe cyclique C, (la diagonale X n’étant pas identifiée 
au point base). La p-puissance réduite g ne se factorise pas en général par 
L(CP,(X)) a homotopie prés, sauf pour R = Z et X connexe. En effet, dans 
ce cas, on peut remplacer L(X) par le produit symétrique infini de la réalisation 
géométrique |X| de X [1]. La p-puissance réduite peut étre alors définie par la 
formule suivante? 


w= Yo o'Wip Nvin No AYi,)- 
1 (Cie oeaen ts) 


2 3; 4 = (Yi, Y2;---Yn:-**) appartenant au produit symétrique infini de 


|X| 
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Dans cette formule, y’ : |X|’? —>+ CP,(|X|) est lapplication quotient, les 
indices ig ne sont pas tous égaux et (71,i2,...,ip) désigne la classe de la 
suite (i1,%2,...,ip) modulo l’action du groupe cyclique C,. En réalisant 
géométriquement les ensembles simpliciaux (ce qui nous permet d’identifier X 
a |X| par abus d’écriture), on peut alors démontrer que le diagramme suivant 
est commutatif A homotopie prés (pour R = Z): 


L(CP}(X)) 


a, 


L(X) & L(CPp(X)) 


Considérons le cas particulier ot X est la sphére S4. La détermination des ho- 
mologies de C'P,(X) et CP}(X) est alors aisée, car CP,(X) est la suspension 
itérée d’un espace lenticulaire. Par exemple, C P2(S%) (resp. CiPa(S 2)\)mavle 
type d’homotopie de S?**(RP,_1) (resp. S91 (RP,_1)V S97"). Si R= Z/pet 
si une base des espaces vectoriels d’homologie de CP,(X) ou CPH(X) a coef- 
ficients dans Z/p est choisie, les espaces L(CP,(X )) et L(CP}(X)) s’écrivent 
comme des produits d’espaces d’Eilenberg-Mac Lane explicites (cf. Pannexe 
Gy 

Pour p impair, |’application g ci-dessus détermine ainsi une opération co- 
homologique 


K(Z/p,q) — K(Z/p,4 + 1) x K(Z/p,q + 2) x +> x K(Z/p, Pa) 
dont les composantes K(Z/p,q) —* K(Z/p,q + 2j) sont en fait nulles pour 
4 #i(p—1) et égales® aux opérations de Steenrod P* pour j = i(p — 1). Pour 
p = 2, Vapplication 
ake ok) (ONE) 
induit une opération cohomologique 
K(Z/2,q) —> K(Z/2,4+ 1) x K(Z/2,q+ 2) x -+- x K(Z/2, 29) 


dont les composantes K(Z/2,q) —* K(Z/2,4 +3 ) sont les carrés de Steenrod 
Sqi. Par ailleurs, considérons la cofibration homotopique 


BCt AX — EC} No, X"? — CP} (X) 


ou BCH (resp. ECF) est espace classifiant du groupe C, (resp. son fibré 
universel) auquel on a ajouté un point en dehors. L’application du produit 


Pe ee ee... a 
3 3 une normalisation pres ; cf. 2.6 et [3]. 
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cyclique normalisé CE (X) dans la suspension de Boy A X (suite de Puppe) 
définit un homomorphisme L(CP+(X)) — L(S' A BC} AX). L’application 


composée L(X) £4 L(CP}(X)) —+ L(S! A BC+ A X), induit (en lui appli- 
quant le foncteur 7, avec n > 1) des opérations de Steenrod en homologie (cf. 
2.6 et 2.9) 


HS) also BCH x) 


pour tout anneau commutatif de coefficients. En homologie mod. p, ces 
opérations sont duales des opérations de Steenrod usuelles (a une normali- 
sation pres). 

Ce qui précéde est détaillé dans les §1 et 2 de cet article. Dans le 
§3 (largement conjectural), nous généralisons les considérations précédentes 
en remplagant le groupe cyclique par un sous-groupe quelconque du groupe 
symétrique ©, en particulier un sous-groupe de Sylow. Enfin, dans le 84, une 
autre généralisation est présentée dans le cadre de la cohomotopie: Vanalogue 
de lespace L(X) (pour R = Z) et X conneze est Q(X) = OF 5 (x) On 
utilise ici une description de cet espace détaillée dans [6] par exemple, ainsi 
que quelques résultats de J. Lannes [4]. Ce §4 n’est pas entiérement origi- 
nal; il recoupe des travaux antérieurs sur les invariants de J ames—Hopf. Citons 
notamment un article de N. Kuhn en relation avec le sujet (Transactions Amer- 
ican Math. Society, 283, 1984, pp. 303-313). 

Enfin, trois annexes sont consacrées & la démonstration de résultats tech- 
niques (cohomologie bivariante, transfert et espaces lenticulaires, etc....), dont 
auteur n’a pas trouvé trace sous cette forme dans la littérature. Ils permet- 
tent notamment de mieux conceptualiser les théorémes démontrés dans les 
deux premiers paragraphes. 


Remerciements. Je remercie W. Dwyer, N. Kuhn, J. Lannes, H. Miller et 
C. Mouét pour des commentaires fort utiles apres une premiére rédaction de 
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Table des matiéres 
1. Produit cyclique d’espaces et puissance réduite 
2. Définition des opérations de Steenrod par la puissance réduite mod p 
3. Généralisation 
4. Opérations de Steenrod en cohomotopie 
Annexe A. Cohomologie bivariante et isomorphisme de Thom 
Annexe B. Transfert et espaces lenticulaires 
Annexe C. Type d’homotopie des groupes abéliens simpliciaux 
Références 


Les notations suivantes sont fréquemment utilisées dans cet article: si G est 
un groupe discret et X un G-espace, Xic est l’espace de Borel associé, soit 
Xng = EG xg X,EG désignant le G-fibré principal universel sur BG; Xpc 
est le “quotient homotopique” de X par G. De méme, X*© désigne l’ensemble 
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des “points fixes homotopiques” , c’est-a-dire l’espace S(€) des sections du fibré 
€= EGxcX sur BG. Les espaces Xg et X © sont respectivement le quotient 
X/G et l’espace des points fixes de X par l’action de G. On a évidemment des 
applications X,g —> XG et X G __, XhG_ Elles ne sont pas des équivalences 
d’homotopie en général. 


1. Produit cyclique d’espaces et puissance réduite 


1.1. Commencons ce paragraphe par quelques considérations classiques sur 
le transfert en homologie et en cohomologie usuelles. Soit G un groupe fini 
opérant & droite sur un ensemble simplicial T’ et: soit T/G l'ensemble quo- 
tient (l’action de G n’est pas nécessairement libre). Un homomorphisme de 
“transfert” 


Pe (Cy a 


est alors défini sur les éléments de base en posant #(t) = digcc #7, pour t € 


T/G, t relevé quelconque de f, # désignant le transformé de t par g. En fait, 
cette formule (qui est indépendante du choix de #) montre que l’image de # 
est contenue dans L(T)°, ensemble des points fixes de L(T) par l’action de 
G. Dans le cadre topologique, X étant connexe, L(X) peut étre remplacé par 
le produit symétrique infini (pour R = Z): il convient de remarquer que w 
est alors continue, induite en fait par une application continue de T/G dans 


Ee 
1.2. Considérons la somme directe 
A(T) = L(T?) ® L(T/G) . 


Une application (notée encore ) de A(T) dans L(T)® est définie en posant 


(to,t)=to+ >? . 


gEG 
Si T et U sont deux G-ensembles pointés, le “cup-produit” 
A(T) \A(WU) — A(T AU) 


associe aux deux couples (to, t) et (uo, u) le couple (wo, w) suivant: 
1) Wo = to A Uo 
2) w est la classe dans L((T,U)/G) de w dans L(T,U) défini par 


oy [2 A.uo) + (to A a9) + (# Aw)] - 
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Un accouplement 
RUBS CO) ORIN 


est défini aussi de maniére évidente. Enfin, si s : BG —+ EG xg T (resp. 
s': BG —+ EG xg) est une section de la fibration EG xg T — BG (resp. 
EG xg U —> BG), le “cup-produit” de s et s’ est la section ur (i,s As’), 
ol ut (u, s) (resp. u+> (i, s’)) est une section de la premiére fibration (resp. 
la seconde). Il est clair que les applications A(T) —> L(T)© et L(T)° — 
L(T)"© sont compatibles avec ces structures “multiplicatives.” 


1.3. Voici une variante de ce qui précéde lorsque p = 0 dans R, p étant l’ordre 
du groupe G. Pour un ensemble T, posons de maniére générale Tp = (i Je 
Le transfert précédent induit alors un homomorphisme 


Cy Oy =O 


En effet, si zr € L(T°), ou T° est considéré comme sous-ensemble de T, son 
transfert est un multiple de p, donc est nul dans L(T). Si on pose de méme 


Ao(T) = L(T*) & L(To/G) 
Yhomomorphisme de A(T) dans L(T)© induit un homomorphisme 
Ao(T) — L{T)® . 


Si U est un deuxiéme G-ensemble, le “cup-produit” A(T’) A A(U) — 
A(T AU) défini plus haut induit un accouplement analogue sur les ensembles 
Ao compatible avec l"homomorphisme de transfert Ao(T’) —> L(T)? comme 
il a été explicité en 1.2. 


1.4. Si X est un ensemble pointé, la “puissance p®™©” est une application 
définie par la formule suivante, déja utilisée en [3]: 


Pp OS dizi) = ss Oe, ae Oe ele A Lig??? A Lin 5 (1) 


41 ,42,...jtp 


Soient maintenant X et Y deux ensembles pointés. On définit un accouplement 
(ou produit) R-bilinéaire: 


L(X*?) A L(Y?) —s L((X AY)*?) , 


C’est celui associé & laccouplement évident d’ensembles X/? A VAs ae 
(X AY)*?. Ces définitions étant posées, nous avons |’identité 


Pp(Z,Y¥) = Yp(x) pp(y) 
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avec des notations évidentes. Si X est une sphére S? et Y une sphere St, 
celle-ci doit étre légerement modifiée par un automorphisme de sP(a+') car le 
“produit” 


L(S?2) A (ED) = L(sPat9)) 
peut étre défini indépendamment de la décomposition de chacune des spheres 


en p facteurs. Cet automorphisme est de degré (yey 2, 


1.5. Considérons maintenant le quotient de X’? par l’action du groupe cy- 
clique C, = Z/p opérant par permutation circulaire des facteurs, la diagonale 
X étant identifiée au point base. L’ensemble obtenu est le p-produit cy- 
clique normalisé de X; il est noté Cle (X). Le groupe cyclique Cp opére 
de méme sur l’ensemble des suites (i1,/2,...,ip). On note (ij,i2,...,%p) la 
classe de (11, %2,..., ip) dans l’ensemble quotient. Si p est un nombre premier’, 
Vapplication composée 


hp  L(X) —> L(X"*) —9 L(OP#(X)) 
s’écrit alors® 


Vp oo dizi) =p > ae peas NipZiz A Lig’ A Lin : 


Gite ip) 
Cette identité permet de décrire la p-puissance réduite 
go: L(x) — L(C P23) 


par la formule suivante: 
¢) om dizi) = Ss; INS eS mr NG Da, J\ BBs, °° ° A Xi, 
(a1 ,82,-+-s¢p) 


avec les conventions précisées dans la Note 5 (le point base étant toujours 
identifié & 0). Notons A}(X) lensemble Ao(X*?) = L(X) @ L(CPF(X)) 


(avec les notations de 1.3) et considérons la composition suivante: 
4: At (X) > L(X%P)% — U(X’? 2 


ou la premiére application est le transfert (chle3). 


1.6 Théoréme. L’application y est multiplicative: on al identité y(z-t) = 
y(z)- y(t) pour z € AF(X) ett e APY). 


4 Le cas ou p est non premier sera envisagé au §3. 
5 Si cette convention ne préte pas 4 confusion, on écrira de la méme maniere 


un élément de L(X”*) et sa classe dans L(CP;'(X)). 
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Démonstration. C’est une conséquence immédiate des considérations 
générales de 1.2-4. 


1.7. Supposons maintenant que p = 0 dans R. Pour x = >> A;z; € L(X), 
posons Z(p) = ))(Ai)?ai. L’application 6 : L(X) —> L(X) ® L(CP}(X)) = 
A} (X) définie par G(x) = (xp), p(x)) est alors compatible avec la structure 
multiplicative sur le deuxiéme groupe définie en 1.2 et 1.3 (il convient de poser 
T= xX’?,U =Y”?, G =C, avec les notations de 1.2). De maniére précise, cal- 
culons 9 (()> 821) - (90 H74;)] = POD Am;r: A Y;). Il vient le développement 
suivant sur chaque simplexe: 


9 (> rif jTi \ uj) = Netty Ae, (Ben Ag, A -A(@i, AY) 
((41,91)--+(tp,5p)) 


ott la somme est étendue a toutes les suites de couples (ta, ja) non tous égaux 
(le groupe C, opérant ainsi avec des orbites de cardinal p). D’aprés les con- 
sidérations générales de 1.2., les éléments de cette somme se répartissent en 3 
types (cf. plus généralement le §3): 


1. Les i, et les jg ne sont pas tous égaux; on trouve alors 


99 (~ dias) ‘9 Oe 1345) 


pour le produit défini en 1.2. 


2. Les tq sont égaux et les jg ne sont pas tous égaux; il vient alors 


Yo Os)PHar Mig (EA YA) AoA (te A Y5,) - 
(i,71,32,---,5p)) 


Cette derniére expression sera notée lq) Cy 


3. Les jg sont égaux et les i, ne sont pas tous égaux; on trouve alors 


iz + Aig (Hz)? (Zi, A YZ) A+++ A (2, A Yy;) 
(Gatto a) 


expression qui sera notée de méme (zx) - yp). 
En résumé, nous obtenons ainsi le théoréme suivant: 


1.8 Théoréme. Supposons que p= 0 dans R. Alors Vapplication 
p: L(x) — L(x) ® L(CPY(X)) = At (Xx) 
définie par P(x) = (xp), @(z)) vErifie la propriété suivante 


P(zy) = P(r) p(y) 
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pour x € L(X) ety € L(Y), les deux membres de la formule appartenant a 
At(X AY), pour la structure multiplicative définie en 1.2 et 1.3. 


Considérons le cas particuler oi X est la sphére S?. Alors X’? = S?4, le 
groupe cyclique C, opérant de maniére naturelle sur S??7. Le lemme suivant 
est évident (remarquer que S?? est le compactifié d’Alexandrof de R?*): 


1.9 Lemme. L’espace quotient S?1/C, s’identifie 4 la (q+ 1)®™° suspension 
(non réduite) de l’espace lenticulaire SP4~1~'/Cp, ot Cp opére sur Ogee 
par Vinvolution antipodique si p = 2 et grace a la représentation induite sur 
CxP-1)/2 nar p@® p? @-:-® ees ot p est la représentation usuelle de 
Z/p = bp dans C*, si p est impair. 


1.10. Remarque. Si X est une sphére, il en résulte que l’application canonique 
de X dans CP,(X) est homotopiquement triviale. Donc CPt (X) a le type 
dhomotopie du “wedge” SX V CP,(X), ot SX désigne la suspension de X. 
En particulier, CP{(S%) a le type d’homotopie de S?*? v S1*1(RP,-1). Par 
conséquent, si R = Z/2, l’espace vectoriel simplicial L(C P; (S*)) est le produit 
suivant d’espaces d’Bilenberg—Mac Lane®: 


K(Z/2,q+1) x K(Z/2,q +2) x --- x K(Z/2, 2g) . 


1.11 Proposition. Soit R = Z/2 et soient P” et P™ deux quotients de S” et 
S™, compactifiés d’Alexandrof de R” et R™, par une action linéaire du groupe 
G=Cy,. Alors l’application 


LE LP) 


définie en 1.2 induit le cup-produit usuel sur les espaces d’Eilenberg—Mac 
Lane K(Z/2,a), composantes des espaces idee yet 


Démonstration. Les espaces P” et P™ sont des suspensions itérées d’espaces 
projectifs réels et application est déduite d’un accouplement 


g?*1(RP,_1) A S**1(RP,-1) — L(ST*'(RPp40-1)) 


avecr+q=nett+s =m. Si q est pair, HAS OCRP1)ye 22 et 
application “derniére cellule” de S*t+1(RP,_1) dans S” est de degré un. On 
peut aussi remarquer qu’une autre interprétation de l’application “derniére cel- 
lule” L(RP,) —> L(S*) est le transfert mod. 2 qui est de degré 1 mod. 2. Ceci 
est clair si g est impair (car l’action de Z/2 est compatible avec l’orientation). 


—————————— 
6 La base de l’homologie de C’P;'(S7) est évidente (cf. l’annexe C). 
7 Chaque espace d’homologie étant de dimension un, il n’y a pas d’ambiguité 
sur le choix de la base (cf. encore l’annexe C). 
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Dans le cas général, il suffit d’écrire le diagramme commutatif 


eet ees sihee —S S4 


I I I 


So es ee oe 


en remarquant que, dans tous les cas, |’application RP, —> S% induit un iso- 
morphisme sur les groupes d’homologie 4 coefficients dans Z/2. Le diagramme 
commutatif suivant (ot les fléches verticales sont canoniquement scindées) per- 
met alors de conclure, en raisonnant par récurrence sur la dimension des espaces 
projectifs: 


L(S™*1(RPy-1)) AL (S**4(RPs1)) > L(S™*(RPy45-1)) 


| | 


LS AES ==) — DS ras) 


Revenons maintenant au cup-produit qui nous concerne plus partic- 
uliérement ici® (avec les notations de 1.3): 


INAS) NES Oe G(R) 


1.12 Théoréme. L’accouplement précédent induit le cup-produit usuel au 
niveau des espaces d’Eilenberg—Mac Lane, composantes des espaces Ao(S”). 


Démonstration. D’aprés la proposition 1.11, il induit déja les accouplements 
usuels 


K(Z/2,q+a)A K(Z/2,q +0’) — K(Z/2,q+q +a+a’), 


sauf peut-étre pour a ou a’ = 0 ou 1. Dans ce cas, utilisons le transfert 
Ao(S”) —> L(S”)?©2 décrit en 1.1 et 1.2. Puisqu’il est compatible avec les 
structures multiplicatives d’apres 1.3, il suffit de montrer qu’il induit un iso- 
morphisme sur les groupes d’homotopie 7, et 7441 & Z/2. Pour le premier 
groupe, ceci résulte immédiatement de [3] 1.15. Pour le second, considérons la 
premiére opération cohomologique K(Z/2,q) —+ K(Z/2,q+ 1) induite par la 
puissance 2'*™° et qui résulte de la décomposition de L($?9)h©2 en un produit 
d’espaces d’Hilenberg—Mac Lane (cf. l’annexe C). Cette opération se factorise 
évidemment par jas (S%) grace & la 2'*™° puissance réduite. I] suffit donc de 
montrer que l’application K(Z/2, q) —> K(Z/2,q+1) n’est pas nulle. Puisque 
c’est ’homomorphisme de Bockstein Sq! d’aprés [3] 1.15, la démonstration du 
théoréeme est achevée (cf. aussi l’annexe B). 


8 De maniére générale, rappelons que pour tout G-espace X, on pose 


Ao(X) = L(X°%) ® L(Xo/G) avec Xo = X/XS. 
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1.13 Théoréme. Soit p un nombre premier impair, et soient P” et P™ 
deux quotients de S” et S™, compactifiés d’Alexandrof de R” et R™, par une 
action linéaire du groupe C,, le sous-espace des points fixes étant de dimension 
strictement positive dans les deux cas. L’application 


A(s") AA(S™) —> A(s"*”) 


définie en 1.3 (pour R = Z) induit alors le cup-produit usuel sur les espaces 
d’Eilenberg-Mac Lane facteurs des espaces A (compte tenu de diverses nor- 
malisations précisées plus loin.) 


Démonstration. D’aprés l’annexe B, avec les notations de 1.3, on sait que le 
transfert 


L(S"/Cp) —> L(S")* 


induit un isomorphisme sur les groupes d’homotopie 7; sii > q+1, q désignant 
la dimension de l’espace des points fixes. Par ailleurs, d’aprés 1.3, il est com- 
patible avec les structures multiplicatives et nous avons déterminé en [3] la 
structure multiplicative des L(S ™\hCpy Plus précisément, la composante neu- 
tre de L(S”)"© ale type d’homotopie d’un produit d’espaces d’Eilenberg—Mac 
Lane K(Z,n) x [], K(Z/p,n — 2r) avec r > 0. Pour n+ j pair, le choix des 
générateurs des groupes d’homotopie 7;(L(S"/ Cp)) = 75 (S(")") équivaut & 
celui des groupes de cohomologie H "+3(BC,; Z) du groupe cyclique Cp, choix 
explicité dans la Note 1 de [3]. 

Pour étre complet, il convient de préciser le choix du générateur de 
™q(L(S%)). Pour que celui-ci soit compatible avec les structures multiplicatives, 
il doit correspondre au générateur naturel de aL (Since) cs es Ce 2) 
par ’homomorphisme de transfert 


A(S") = L(S*%) @ L(S"/G) — L(S")** . 
Un cas particuligrement important est celui ou S” = SP4 (produit contracté de 
p facteurs $7). Dans ce cas, on a vu au 81 de [3] que Papplication L(S7) — 


L(S?7)"C» induit sur les générateurs “naturels” des groupes d’homotopie 7q la 
multiplication par (—1)°(m!)? avec m = (p—1)/2 et s= m(q? — q)/2. 


1.14 Remarque. Le cup-produit 
E(GP; (5%) \ L(CPz ($7 )) — L(CP3(st*)) 
introduit un signe au niveau des générateurs canoniques, car l’application 
gpa, gpa —, gp(ata'’) 


est de degré(—1)#47'?(?-))/? (comparer avec (BS1-7): 
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2. Définition des opérations de Steenrod 
par la puissance réduite mod. p 


2.1. Soit S ensemble des suites non constantes (€1, €2,...,€p) telles que €; = 0 
ou 1. Le groupe cyclique C, opére transitivement sur S avec des orbites de 
cardinal p. Soit S l’ensemble quotient et soit s : S —> S une section arbitraire. 
On définit une application 


O, : L(X) x L(X) —> L(X”’?) 
par la formule suivante (ott on pose s(%) = (€1,€2,...,€p) pour u € S): 
Op (SL AMeP DI Nat) = DD IAB AEaEe Aa A Ake 
aCe 
Si p = 2 par exemple, pour un choix évident de s, la formule s’écrit simplement 
2 O23 d229, Nat) = = 
(¢,9) 


c’est-a-dire comme le cup-produit usuel. Si p = 3, on peut choisir la formule 
suivante: 


Cu 32 oe) ye ee 


(car (0,1,1) et (0,0,1) sont deux représentants de 5). Désignons par v : 
L(X’?) —+ L(CP+(X)) Vhomomorphisme induit par l’application quotient 
he er Xx) 


2.2 Théoréme. Soit 9 : L(X) — OT EO) TO. définie par la 
p-puissance réduite (cf. 1.5) et soient x° = > \9x° et zg! = Ajai deux 
éléments de L(X). On a alors la formule suivante: 


p(2" +2!) = g(2°) + (2) + o(O,(2°, 21) . 

Démonstration. Le développement de g(x° + x!) donne la somme suivante: 
0 1 e1\e Cp e € 

pla") + e(z')+ Do A Ag Ae Bi, NB AGE AW Aas? 


ou (€;) est une suite non constante et ow le > est évidemment égal A 
v(O,(x°, z')). Le théoréme en résulte aussitét. 
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2.3. Ce dernier résultat a une conséquence importante que nous allons décrire 
grace au diagramme cocartésien homotopique 


BC, x X/BC, (Once?) BC, 


[ | 


x > CPX) 
De celui-ci se déduit une application canonique bien définie 4 homotopie prés 
CP,(X) —+ 81 A(BCt AX) = S* A(BC, x X/BCp) , 


ou B Ce désigne l’espace classifiant BC, auquel on a ajouté un point en dehors. 
Notons que cette application se factorise par CP} (X) = CP,(X)/X (cf. aussi 
2.8). En désignant par g* lapplication composée de la puissance réduite 
go: L(X) —> L(CP}(X)) et de ’homomorphisme L(CP}(X)) —> L(s' A 
BC+ A X) induit par application précédente, nous pouvons écrire la formule 
d’additivité suivante (A homotopie prés): 


p* (2° +21) = p* (x9) + p*(z") 


(car application (2°, 2!) + p#(x° + at) — yg (2°) + p*(x') se factorise par 
L(X*?) & homotopie prés). En particulier, si X est une sphére S? et si 
L(CPF(S%)) est décomposé en un produit d’espaces d’Hilenberg-Mac Lane 
(cf. annexe C), les applications L(S?) —> L(CRS (33) a2 / par) 
qu’on en déduit sont homotopiquement additives pour g+r < pq. L’application 
puissance p*™e de L(S?) = K(Z/p,q) dans L(S??) = K(Z/p, pq) induit aussi 
une opération additive au niveau de la cohomologie pour des raisons évidentes. 
Nous allons déterminer ces opérations en commengant par le cas p = Be 


2.4 Théoréme. Soit R= Z/2. L’application 
p : K(Z/2,q) = L(8") — Az (S*) 
ou At (S7) = L(S%) ® (G2 (RP) vs) 
2K (7/24) x Kia 2.¢ + K(Z/2,q¢ +2) x --: x K(Z/2,2q) 
induit Vopération cohomologique somme des carrés de Steenrod 


dim(z) 


p(e)= D7 Sa'(z). 


Démonstration. Les opérations cohomologiques additives Sq’ sont car- 
actérisées par les axiomes suivants: 
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Sq =1. 

Sordi) —og (a) — 

. Sqi(z) =0 sii > dim(z). 

5) Ca) = Daye Gl 27). 

Puisque |’opération cohomologique w(x) vérifie les mémes axiomes d’aprés ce 
qui précede, on a bien (x) = v(x), ce qui démontre le théoréme. Une autre 


démonstration peut étre aisément construite a partir de [3] §1 et de l’annexe 


B.4. 


eB WN 


2.5. Soit p un nombre premier impair tel que p = 0 dans R et soit 7 
endomorphisme idempotent de L(CP}(X)) défini par la formule suivante: 


1 
Giese = Nan) c= SS Geis ere 
" a€Gp/Cp 


o parcourant |’ensemble des classes 4 gauche de |’ensemble quotient Gyc.. i 
est clair que l’image de l’application composée 


L(X) — L(X) @ L(CP}(X)) 4 L(x) e@ L(CP#(X)) 


est contenue dans le sous-groupe invariant par 7. Cette remarque implique le 
résultat suivant: 


2.6 Théoréme. Soit p un nombre premier impair et soit R = Z/p. 
L’application 


K(Z/p,q) = L(S?) —+ AF (8%) , 
ot At (S$?) = L(S%) ® L(S%*! v (S?9/C,)) 
~ K(Z/p,q) x K(2Z/p,q +1) x K(Z/p,q+ 2) x --- x K(Z/p, pq) 
induit des opérations cohomologiques 
H°(X;Z/p) — H1°"(X;Z/p) . 


Celles-ci sont triviales sir #0 mod. p—1. Sir =0 mod. p—1, Vopération 
cohomologique obtenue 


PY: HX; Z/p) — HU*C-) (x; 7 /p) 
est la puissance réduite de Steenrod (cf. [5] et [3], 1.14), multipliée par le facteur 


de normalisation (—1)"(m!)? ezplicité en [3] 1.14, m étant égal (p —1)/2 et 
r=t-m+m(q? —q)/2). 
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Démonstration. Ce théoréme résulte immédiatement de l’annexe B.1-2 et 
des considérations développées dans [3] §1 (cf. 1.13 plus particuliérement). 


2.7 Remarque. Ce qui précéde montre que la puissance réduite L(X) — 
L(CP} (X)) ne se factorise pas par L(CP}(X)) en général, CP(X) désignant 
lespace de Borel EC, xc, X“?, application L(EC) xo, X’?) —+ L(CPF(X)) 
étant induite par la deuxiéme projection. Par contre, nous verrons au §4 que 
les opérations de Steenrod en cohomotopie (définies comme applications de 
Q(X) dans Q(CP,(X)) se factorisent par Q(C-P*(X)) (cf. aussi 2.11). 


2.8. Réécrivons la cofibration homotopique établie en 2.3: 
BCs AX — ee Lom XxX’? —; CPs (X) ; 


BC} (resp. EC) désignant l’espace classifiant (resp. le fibré principal uni- 
versel) du groupe cyclique Cp auquel on a ajouté un point en dehors. De 
cette cofibration, on déduit une application CP+(X) — $ 'A BCH AX (suite 
de Puppe). En lui appliquant le foncteur L et en composant par la puis- 
sance réduite L(X) —+ L(CP+(X)) comme en 2.3, on obtient une application 
canonique 


L(X) — L(S'A BCT AX). 


Celle-ci permet de retrouver les opérations de Steenrod d’une manieére 
différente pour un anneau commutatif de coefficients R quelconque. De maniere 
plus précise, définissons une théorie de la cohomologie bivariante sur 
les espaces pointés X et Y en posant HH(X,Y) = [Y,L(X)] (ensemble des 
classes d’homotopie d’applications pointées; plus de détails sont proposés dans 
annexe A). Si_X (resp. Y) est une sphére S”, on trouve la cohomologie réduite 
H"(Y) (resp. ’homologie réduite H,,(X)). Les considérations précédentes per- 
mettent de définir une application 


HH(X,Y) — HH(S!A BCt A X,Y). 


En particulier, pour Y = S”, on en déduit des opérations de Steenrod en 
homologie 


Hex) — (8 ABCe A Xx) 


Vhomologie étant prise 4 coefficients dans un anneau commutatif arbitraire R. 
Celles-ci sont & comparer aux opérations cohomologiques 


H"(X; R) —+ H"?(X x BGp; Re) 


ot R- est le systéme local induit par la signature sur 6, a valeurs dans 
+1 Cc R, définies dans [3]. Si R = Z/p, la formule de Kimneth per- 
met décrire les opérations de Steenrod homologiques comme morphismes 
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H,(X) — @, Hn_-(X) (les générateurs de ’homologie du groupe discret 
C, étant choisis de maniére explicite: cf. la Note 1 de [3]). 


2.9 Théoréme. Soitn—r> 0. Si R = Z/p, le morphisme H,(X) — 
H,-,(X) défini ci-dessus est dual de V’opération de Steenrod non normalisée 
D,: H""(X) — H"(X) définie en [3] §1.3 et 1.5 avec s = p(n —r) —r. 


Démonstration. Ecrivons L(S' A BC) comme somme des groupes abéliens 
simpliciaux €)°-_, L(S*) (& homotopie prés; cf. l’annexe C). On a alors 
L(S' A BC} AX) » @e, L(S*) ® L(X). Soit maintenant u € H%(X), 
représentée par une application f, : X —> L(S%). L’opération de Steenrod 
L(S%) — L(S* a BCS A S41) = @e_, L(S$*) ® L(S%) induit une applica- 
tion L(S7) —+ L(S% A S%), donc, par composition avec f,, une application 
X — L(S%7 A S%), c’est-a-dire un élément P,(u) € HIt%(X). La corres- 
pondance u +> P,(u) définit Popération de Steenrod cohomologique D, avec 
s = (p—1)q-a. Par ailleurs, soit v € Hj+.(X), représentée par une appli- 
cation S7** — [(X). Alors, l’opération homologique associée & a, soit Qa, 
s’obtient par la composition des morphismes suivants: 


St Ve LO NS) 1G) @ (52) 


Ona ams (ye Hope ees (CX). 
Le produit scalaire (v, P.(u)) est le degré de la composition 


Sit _, I(X) — L(S%) — L(S7 A S*) 
tandis que le produit scalaire (Q.(v), u) est le degré de la composition 
Sit _, L(X A 8%) —+ L(S% A 8%) 
ou encore 
site —_, L(X) — L(X AS*) — L(S4 A S*) . 
Il suffit donc de vérifier la commutativité du diagramme suivant: 


L(X) — L(S?) 


| | 


L(X AS*) ——+ L(S7aS*) 


Elle résulte de la naturalité de l’application L(X) — L(X A S$). 


2.10 Remarque. Soit z un élément du noyau de l’application de Steenrod en 
homologie H,,(X) —> H,,(S'ABC} AX). Grace & la cofibration homotopique 


BO} AX —+ EC# nc, XP —+ CPF (X) 
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on peut appliquer 4 x une opération de Steenrod secondaire. Le résultat ap- 
partient au conoyau de l’application H,(BC{ A X) — H,(EC} Ac, X”?), 
soit H,(BC, x X/BC,) — H,(EC, xc, X*?)/BC,). Ce cas se présente 
par exemple si x appartient 4 l’image de ’homomorphisme de Hurewicz 


ae) 2H, |X) (ci wets). 


2.11 Remarque. Dans. ce paragraphe, a |’exception des derniéres con- 
sidérations, nous nous sommes restreints 4 R = Z/p. Cependant, la 2-puissance 
réduite par exemple (pour R = Z) définit une application 


K(Z,q) = L(S?) —+A2(S%) & K(Z,q) x K(Z/2,q +2) x ++ 
... x K(Z/2,q +4) x K(Z/2,2q) 


d’aprés l’annexe C. Cette extension des définitions est néanmoins illusoire. 
En effet, d’aprés la fin de l’annexe C, l’application canonique At ($7) — 
St (S%)/2 induit sur les espaces d’Kilenberg~Mac Lane facteurs des A le pro- 
duit de application identique par homomorphisme de Bockstein. Puisque 
Sq2*+1 = Sq1Sq??, les opérations cohomologiques ainsi définies sur K (Z, q) 
se déduisent de celles définies sur K(Z/2,q) par l’application composée 


K(Z,q) — K(Z/2,q) = K(Z/2,q-+ 24). 


3. Généralisation 


3.1. Nous nous proposons de généraliser partiellement les considérations des 
deux paragraphes précédents en remplagant le groupe cyclique Cp, p premier, 
par un sous-groupe G du groupe symétrique 6, 7 étant un entier quelconque. 
Pour simplifier les considérations du début, nous supposerons que R=Z, ce 
qui permet de remplacer L(X), X connexe pointé, par le produit symétrique 
infini de X, d’un maniement plus simple et plus géométrique [1]. 


Si I = (i1,72,-.--, in) est une suite d’entiers et si g € G, nous faisons opérer 
g 4 droite sur cette suite en posant I9 = (ig(1), 49(2)2++- ,1g(n)). De maniére par- 
alléle, g opére & droite sur X”” par (xi, Z2,.-- pte)o =e eos eta) 


De maniére générale, si U est un G-espace et si H est un sous-groupe de G, on 
désigne par U\#) la réunion des sous-espaces de U formé des éléments invari- 
ants par H ou par un sous-groupe conjugué gHg~'. En particulier, considérons 
la réunion 


1 


(XArVED = Wiese ees 


9 


évidemment invariante par ]’action de G. L’espace quotient (Xx An\(H) /G, noté 
simplement X 77), va jouer le réle du produit cyclique de X étudié dans le 


paragraphe précédent. 
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De maniére précise, soit c = )7;",2; un élément de L(X) (puissance 
symétrique infinie de X). La puissance ni*™, soit x”, appartient A L(X%") 
et s’écrit comme la somme de m” termes x;, A zi, A+++ A Zi,, somme qui 
peut se décomposer de la maniére suivante suivant le “type” de la suite J = 
(i1,72,..-,%n). Le type (H) est formé de suites I dont le stabilisateur est un 
sous-groupe conjugué de H. Leur orbite (71, 72,...,in) est done de cardinal 
|G|/|H|. Bien entendu, le type (H) ne dépend que de la classe de conjugaison 
(notée aussi (H)) de H dans G. Si G = C,, p premier par exemple, il n’y a 
que deux types: le groupe G lui-méme (il s’agit alors des suites constantes) et 
le groupe trivial (les indices ig ne sont pas tous égaux). 

A chaque type (H), on associe la “H-puissance réduite” de x comme 
lexpression suivante appartenant & L(X(q)) := L((X*")“)/G): 


) Hes, aba, Mooe iN ais 
(Cl ggsnetin) 


Cette somme est étendue aux suites de type (H). La “puissance réduite totale” 
est l’application 


OO) BD L(X x) 
(H) 


ou chaque composante de 9, soit. 


correspond a la somme explicitée ci-dessus. En particulier, pour H réduit a 
Pélément neutre 0, on en déduit une application 


po L(X) LX /G) = L(X) . 
Un homomorphisme de “transfert” 
ou : L(X (uy) — L(x") 


est défini en associant 4 la classe de élément x = 21 A zo A--- A Zn dans 
(X4%")9H9"" Ja somme suivante 


J ene 


t€G/H’ 


ou t parcourt l’ensemble des classes & gauche de G par H’ = gHg™!. Cette 
somme est bien invariante par G. Elle est indépendante du choix de g@: si? 
est le cardinal de H, il suffit de remarquer que r - y est la somme des reat 
parcourant le groupe G. Puisque L(X’”) est un Z-module libre, |’élément y 
est bien défini par la formule ci-dessus. 
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3.2 Théoréme. L’application composée 


L(X) > DH L(X uy) wh. Tag? LX)” 
(H) 


coincide avec la puissance ni*™* équivariante définie dans [3]. 


Démonstration. II suffit de décomposer (}> x;)” suivant les différentes classes 
(H) de sous-groupes de G. 


3.3. Soit K un deuxiéme sous-groupe de G tel que H C K. On suppose que 
G opére A droite sur deux espaces pointés U et V. On peut alors définir de 
maniére générale un “cup-produit” qui généralise celui introduit en 1.2, soit: 


y: L(U\/G) AL(V®)/G) — L(U A Ye Tey 
par la formule suivante 


Tel he) a ey 


geG/K’ 


ot K’ = aKa@ si u € aHa!. Il convient de vérifier que cette somme ne 
dépend que des classes de u et v dans U()/G et V')/G respectivement. Pour 
u ceci est clair, car application g +> 8g induit une translation de G/K' (défini 
par la relation d’équivalence x ~ y ssi zg ty € K’). Par ailleurs, y(u, v?) = 
geG/K* ug? Av et la translation g + gG—* induit un isomorphisme de G/K' 
sur G/K” avec K" = BaKo-18-'. Enfin, la somme définissant (u,v) est 
indépendante du choix de a. En effet, elle peut s’écrire aussi >> 9€G/K" uAvs ; 
= a 

ou encore + >> pee U7 7 étant le cardinal de K. 

3.4. Un exemple intéressant est celui ot G est un p-sous-groupe de Sylow® du 
groupe symétrique 6, avec n =p", p premier. On désigne par S = Spr un tel 
sous-groupe. Considérons le quotient (noté X4") de X%" par V’action de S. 
La puissance réduite (correspondant au sous-groupe trivial H = 0) est alors 
définie comme une application 


po : L(X) — L(X*") 


qui associe A z = )) a; |’élément suivant de HOO 


0(x) = ye eee INL 


(41 ,12,.+-;¢n) 


9 Rappelons que deux sous-groupes de Sylow sont conjugués. 
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Dans cette expression I = (7), %2,...,in) est une suite d’indices sur lesquels le 
groupe 5 opere librement (i.e. (i,(1), t¢(2),--->%g(n)) 7 (41,%2,---in) Si g F € 
et (i1,%2,...,%n) désigne une orbite (gp a été défini dans un contexte plus 
général en 3.1). Si S n’opére pas librement sur la suite I, il existe un élément 
g d’ordre une puissance de p tel que (%9(1),%9(2),---,%g(n)) = (t1, 42,---, in); en 
décomposant la permutation g en cycles, on voit que p indices i, au moins 
sont égaux. 


3.5. Il convient de noter la non-trivialité de l’opération cohomologique 
go : L(X) — L(X%"/S). Celle-ci est sans doute reliée aux compositions 
d’opérations de Steenrod classiques. Vérifions ce fait pour r = 2, soit n = p?. 
Dans ce cas, le groupe S = S,2 est de cardinal p?*?; il est engendré par p cycles 
d’ordre p, soit (1,2,...,p), (p+1,p+2,...,2p),...,(p? —pt+1,...,p?) ainsi 
que par le cycle consistant 4 permuter circulairement les paquets précédents. 
Le fait que S opére librement sur une suite (7), %2,...,%,2) revient donc 4 écrire 
que chaque paquet (71,...,%p), (ip1i,---,%2,), etc.... n’est pas formé de suites 
constantes et que les p paquets ne sont pas tous égaux. Considérons maintenant 
la composition 


HOR) te Oe CIS) = LOS aes EXO) Geox Ce = EOE rs Ce) 

(1) 
ot la deuxiéme application est le transfert (cf. 1.1 et 3.1) et la troisiéme la 
restriction, le groupe C, x C, étant inclus dans S par la permutation simultanée 
des paquets précédents entre eux et dans leur intérieur. Nous pouvons de méme 
composer la puissance de Steenrod L(X) —+ L(X”?)"? avec elle-méme (cf. [3] 
2.11). La différence est la somme )[totsDo,s(z) + t,-toD,,o(x) — totoDo,o(z)] 
avec les notations de [3] 2.11. On en déduit que la composition (1) ci-dessus 
est la somme }/,19,.40 trtsD,,s(x) qui n’est pas triviale en général. 


3.6. Plagons nous maintenant dans le cadre simplicial, ce qui nous permettra 
de considérer un anneau commutatif de base quelconque R. Un élément x de 
I(X) = L(X; R) s’écrit alors comme une somme > 2;z;, 00 les \; sont des 
scalaires appartenant & R (les x; étant distincts). On exprime l’élément g(z) 
de maniere légérement différente comme la somme 


MG At Aig AG, 2a Nl aa ee 


(Gn aocagecn)) 


ou la suite (7, 72,...,%,) est de stabilisateur trivial. Cependant, Vapplication 
ainsi définie n’est pas simpliciale, car le cardinal des orbites par l’action de S = 
Spr peut se modifier par une opération face. Pour remédier A cet inconvenient, 
notons X§” le sous-ensemble de X*” formé des suites x = (fi, 2o,ce ee) 
dont le stabilisateur (pour l’action de S') n’est pas trivial. Ce sous-ensemble 
est invariant par l’action de S. On pose XP" = X/"/S et XA” = DOT GASH 
on interpréte la puissance réduite comme une application 


L(X) —> L(X*"/X)) , 
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elle devient alors simpliciale. Cette définition est bien une généralisation de 


la situation développée dans le paragraphe précédent. En effet, si on choisit 
7 oe Opa ict OC Pax). 


3.7 Théoréme. La puissance réduite définit un morphisme simplicial 
gp: L(X) — L(X™/Xo”) . 


Avec des notations évidentes, elle peut étre définie simplement par la formule 
suivante 


Démonstration. Elle résulte immédiatement des considérations précédentes 
grace a la méthode décrite en 1.5. 


3.8 Proposition. L’espace quotient LOS a le type d’homotopie du quo- 
tient des espaces de Borel correspondant 


BS xs HS <5 Xa 


Démonstration. En effet, l’action de S est libre sur le complémentaire X An _ 
XA". L’espace quotient écrit dans l’énoncé est donc égal & X*" — XG”, car 

0. Pp quotle 1) 
a Set = X67 5. 


3.9. Soit R = Z/p’, s étant un entier arbitraire. Les espaces SO OE 
étant difficiles & décrire directement, nous allons introduire une variante en 
remplacant S par le groupe symétrique G = Gp lui-cméme. De maniere précise, 
définissons un endomorphisme 7 de L(X’”) en posant 


S 
AN tae on) — a > Gey Ae faa) - 
cEG/S 


Alors 7 est idempotent sur L(X/"/S), car c’est la composée 71 072 de deux ap- 
plications telles que 72-71 = Id. D’une part, 7, : L(X*"/G) — L(X*"/S) est 
définie par la méme formule que 7; d’autre part 79 : L(X4"/8) — L(X*"/G) 
est l’application quotient. L’image de la puissance réduite L(X) —> 
L(X"/S) est invariante par T: elle est donc contenue dans L(X*"/G) con- 
sidéré comme sous-groupe de L(X*"/S) par application 71. Seite, la 
réunion des sous-espaces conjugués de X@” par l’action des éléments g de G, 
c’est-A-dire XP” = UgX$"g~?. Alors X}” est la réunion des sous-espaces de 
X*” formé des produits 41 A 22 A-*:AZn, ou p élements xq au moins sont 
égaux comme on le voit en décomposant les permutations en cycles (cf. 3.4). 
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3.10 Théoréme. Soit R = Z/p*, s étant un entier arbitraire avec n = p". 
Alors la n-puissance réduite mod. p* définit une application simpliciale 


L( == EG xe Hee 


ou G = BG, et ou Xf” est la réunion de sous-espaces de X’” formé des produits 
(v1 Arg A---AZpn) dans lesquels p éléments t_ au moins sont égauz. 


Considérons le cas particulier ot: X est un sphére de dimension g. Alors 
X” est le compactifié d’Alexandrof de R’”. Le sous-espace X/” est réunion de 
compactifiés d’Alexandrof de sous-espaces de R%” de dimension m = q(n—p+1) 
qui sont obtenus en choisissant p coordonnées parmi n (il y a donc () tels 
sous-espaces). Nous allons déterminer une partie importante de la topologie 
de l’espace Xj" grace au lemme suivant: 


3.11 Lemme. L’espace Xj" contient en facteur direct (a4 homotopie 
équivariante prés) le bouquet de sphéres V:S™, ot m = q(n —p +1), et od 
t parcourt l’ensemble des parties a p éléments d’un ensemble a n éléments 
(n =p" comme plus haut). 


Démonstration. L’espace Xj” est le compactifié d’Alexandrof de la réunion 
de sous-espaces de R” qui sont de dimension m: chacun de ces sous-espaces est 
Vimage par l’action de G du sous-espace V défini par ensemble des vecteurs 
s’écrivant (11,22,...,2n), ol 2; appartient 4 R% et ob z] = ro =--- = oF 
On définit maintenant une application 


gp:\f/s™ — xi" 


t 
ou chaque fléche S™ —+ Xj est induite par l’inclusion d’un sous-espace. Par 
ailleurs, W9 = (U, 2 ve) NV est un sous-ensemble de codimension > 1 de 
V9. Il existe donc une application de degré un, soit 0, : (V+)? — (Va 


(ot V* désigne la sphére S™, compactifiée d’Alexandrof de R™) telle que que 
4,(W) = {oo}. On définit alors une application équivariante 


p. Xi" — \/s 
t 


comme étant égale a 8, sur chaque cellule de dimension m, {co} ailleurs. I] est 
clair que la composée ~ - y est homotope a4 Videntité de maniére équivariante. 


3.12 Théoreme. Pour i < qn, lhomologie relative H;(X', X/") (4 coeffi- 
cients dans R) contient en facteur direct Hi_m(BK; 2) pote — 6, x G,=, 
est le sous-groupe évident deG = 6, (n=p";m =q(n—p+ 1) 


Démonstration. Cette homologie relative est nulle en degrés > qn. Pour les 
degrés inférieurs a qn, c’est celle de la paire (EG xg X", EG xg Xf"), donc 
de EG xq X{", d’aprés l’isomorphisme de Thom appliqué au fibré EG Max 
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sur BG (ce fibré est orienté si p est impair; si p = 2, il convient d’utiliser des 
coefficients locaux). Par ailleurs, le lemme précédent implique que |’espace 
EG xq X{" contient en facteur direct EG xg Y, ot Y = \/, 5”, t parcourant 
lespace homogéne G/K. D’aprés le lemme de Shapiro et la suite spectrale 
de Serre, on a par ailleurs H;(EG xg Y) = Hi-m(BK), ce qui acheve la 
démonstration du théoréme. 


3.13. Remarque. Le théor’me précédent permet de définir pour tout espace 
T des opérations cohomologiques: 


H4(T; Z/p*) — H*(T; Gi) 
avec n = p’3m = q(n—p+1); i < qgnet ot G; = H*-™(BK; Z/p*), avec 


K = ©, x &,-». Un probleme ouvert est d’étudier leur relation avec les 
opérations de Steenrod usuelles. 


4, Opérations de Steenrod en cohomotopie!® 
4.1. Si X est un espace pointé connere, rappelons d’abord que Q(X) = 
Q°S°X peut étre décrit & partir du groupe symétrique infini de la maniére 
suivante (cf. [6] par exemple). Dans la réunion disjointe des espaces suivants 


Xp = ROM xg X” 


(ot R© désigne la réunion de tous les espaces R"), identifions™ 1’élément de 
Xnim défini par 


(ity, oon ee lg oe RU anne ye oda acai) 
ou * désigne le point base, 4 )’élément de X,, défini par 
Gilet went it oye) . 


Il est alors démontré dans [6] que l’espace ainsi obtenu a le type d’homotopie 
de Q(X). Cet espace peut étre muni d’une loi + de Xn X Xm dans Xnim.- Elle 
arteries oy (Ae) = (jac eens sone) eG) = (tit, ued ts eee Uae) 
la suite 


(t,2) + (u,y) = Ce ase ip pee ea) 


10 Comme il a été signalé dans l’introduction, ce paragraphe ne prétend pas 
étre totalement original. Il est relié 4 de multiples travaux antérieurs sur les in- 
variants de James—Hopf et les scindages de Snaith. Cependant, la comparaison 
avec les opérations de Steenrod semble étre nouvelle. 

11 De maniére générale on désigne par Y() le sous-espace de Y" formé des 
suites (yi, y2,---, Yn) telles que les éléments y; soient distincts. 
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ou les & sont les images des u dans un autre facteur R™. Cette loi + est 
indépendante du choix de cet autre facteur 4 homotopie pres. 

L’élément (t1, t2,...,tnj21,22,---,2n) considéré plus haut sera noté de 
maniére plus suggestive sous la forme 


i) +) + to *2o +: in oy 
ot les t; sont distincts. En particulier, la projection sur le facteur x du couple 


(t,z) avec t = (t), to,...,t,) et @ = (21, %2,...,7,) (ce qui revient, 4 projeter 
R® sur le point base) induit une application 


Q(X) — L(x), 
ou L(X) désigne le produit symétrique infini (ou son groupe symétrisé). Si 
X est la sphere S?% par exemple, nous retrouvons l’application canonique 
Q(S7) — K(Z,q): elle induit ’homomorphisme de Hurewicz cohomologique 
mi(X) —> H4(X) de la cohomotopie stable vers la cohomologie pour tout 
espace connexe X. De maniére duale, en appliquant le foncteur 7, au mor- 


phisme Q(X) — L(X), on trouve l’homomorphisme de Hurewicz homologique 


we (X) — H,(X). 


4.2. Le “cup-produit” classique 
Q(X) AQ(Y) — Q(X AY) 
est induit par l’accouplement Xm A Yn — (X AY)mn défini par 
[(¢i, ra), (ty, €5)] > (tity, i A Zj) 


ou t;t; désigne le couple (t;,t;) € R° PBR©® & R©&. Le résultat final sera noté 
de manieére abrégée 


SS ttj2j A Lj - 


(4,9) 
On définit de méme une opération “puissance pi*™e”: 
$: Q(X) — Q(x’?) 


par la formule 


(t,z) > Ne ti, tig? * ta Vi, AT, A->- AZ, 


(Gipcone en) 


(somme de n? produits). 
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Grace 4 identification de R® 4 R° @R™ ---@R&, le groupe symétrique 
G, opere a droite sur Q(X“?) par la formule suivante (o étant un élément de 


SG): 


ye UG) © Woh 9 UA Nes AC en 


(iPad) 


ee 2 a eta 6 1226). 


(41 ,2,.--,tp) 


4.3. Il est clair que l’image de ¢ est invariante par cette action. La puissance 
gieme par exemple est définie par 


(2, So tity £4 NX; 3 


L’action du groupe Z/2 sur R® @ R© et X AX équivaut 4 permuter (7,7) et 
(j,2), ce qui définit un élément d’ordre 2 dans 6,2. 

Etudions plus en détail l’action de 6, sur Q(X”?) décrite plus haut. Elle 
est en fait le produit de deux actions qui commutent entre elles: celle (notée 
2X) induite par l’action de 6, sur R® = R™@---@R® (p facteurs R®) et une 
deuxiéme (notée 2), induite par l’action de 6, sur X“?, donc sur OO?) 


4.4 Lemme. L’action X, est homotopiquement triviale. 


Démonstration. I] est clair que le modéle homotopique de Q(Y) ne change 
pas si on remplace R® par R© @ (R® ®---@ R®). Sur le premier facteur 
R~,G, opére trivialement (action induite sur Q(Y) notée Xp). Sur le second 
facteur (R® @R®@---@R”~), Gp opére par permutation des facteurs R°. On 
obtient ainsi des équivalences d’homotopie équivariantes 


(Q(Y), Xo) — (Q(Y), to @ M1) — (Q(Y), %) . 


Le lemme en résulte. 


4.5. Ainsi, & condition d’inverser formellement dans la catégorie des fractions 
les équivalences d’homotopie équivariantes, on voit que la puissance p*™ induit 
application suivante (le groupe 6, opérant sur X AP, donc sur Q(X“?), par 
permutation des facteurs): 


TES EO 


Par ailleurs, dans le §2 de [3], nous avons défini une application semblable pour 
les produits symétriques infinis 


pea ye 
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Il est clair que le diagramme suivant commute 


09 $e 


| | 


Li $= EO 2) 


4.6. Restreignons-nous maintenant au sous-groupe C, = Z/p de Gp, p étant 
un nombre premier. Pour un CW complexe Y avec C, action, J. Lannes [4] 
a démontré l’équivalence d’homotopie suivante (a condition de compléter les 
espaces considérés au nombre premier p): 


Q(Y)*? = Q(Y?) x Q(Yac,)?™ 


ou Vic, = Yno,/BC, (cf. les notations dans |’introduction). Appliquons ce 
théoreme a Y = X”?, le groupe C, opérant sur X”? par permutation circulaire 
des facteurs, le premier facteur Q(Y ©?) s’identifiant ainsi A Q(X). Déterminons 
le second facteur du produit précédent si X est la sphere $7, donc Y = S?9, 
le groupe C, opérant sur S?%, considéré comme compactifié d’Alexandrof de 
R??. Si p est impair, Vac, s’identifie a l’espace de Thom du fibré qV sur BC,, 
ot V est le fibré suivant: 


Vite oe Ls? 


(cf. [3], §1.13: 1 désigne le fibré réel trivial de rang un et L le fibré canonoique 
sur BC, qui est de rang complexe 1, donc de rang réel 2). Cet espace de 
Thom est la limite inductive (suivant N) de la q'*™° suspension du compactifié 
d’Alexandrof de (CN ~— C#'?—1)/2)/R. Ici R est la relation d’équivalence sur 
C qui identifie deux suites (x;) et (yi) si yg = Az; avec A; = XA si i > 
q(p — 1)/2 et A; = une puissance de \ comprise entre 1 et (p — 1)/2 pour 
chaque espace C contenu dans C#?—1)/2, La limite inductive s’identifie ainsi a 
S41(BCp/Lq,»), ¢°™* suspension du quotient de l’espace classifiant BC, par un 
espace lenticulaire L,,,. Cet espace lenticulaire L,,, est obtenu comme quotient 
de l’ensemble des vecteurs non nuls de l’espace vectoriel C’ @--- @ Ci(m = 
(p — 1)/2 facteurs C*) par la relation d’équivalence qui identifie les suites 


(G1, 2a, ret er ee SNC va) 


pour 2 racine pie de l’unité et 2; € C%. Le résultat final est donc une 
application “puissance p'*™°” 


Qs) Q(s") x Q(S7(BC,/Lqp)) 


(du moins aprés complétion des espaces considérés en p). En particulier, 
application canonique de Q(S1(BC,/L,,,)) dans Q(S%+1(L,,,)) permet de 
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définir le morphisme suivant apres complétion en p: 
Q(S?) — Q(S* (Ly) - 


4.7. On peut procéder de méme pour p = 2 en remplagant C” par R”. Aprés 
complétion des espaces au nombre 2, on obtient une application 


Q(S7) — Q(S*%) x Q(S*(RPoo/RPg-1)) 5 


ou RP,,-1 (resp. RP.) désigne l’espace projectif réel de dimension n — 1 (resp. 
infinie). Comme en 4.6, cette application se déduit d’un théoreme de J. Lannes 
[4]. Il en résulte une application de Q(S%) dans Q(S?**(RP,-1)), du moins 
apres complétion au nombre 2. 


4.8. Le lecteur attentif aura remarqué des similitudes frappantes entre les 
considérations de 4.6-7 et celles développées dans les paragraphes précédents. 
En particulier, pour tout espace X, se pose le probléme de l’existence d’une 
“puissance réduite:” 


Q(X) — Q(CP,(X)) - 
Pour cela, reconsidérons la puissance pie™e ordinaire 
RO xq, KX —s RM) x5, (XP) 


qui a été définie formellement par la formule suivante 


Res pea. +t Gi, N Vig No ANDi, - 


Le quotient de R® ~ (R®)? par l’action de C, est encore un espace contrac- 
tile. Cependant, les classes des éléments t;, ti, ---ti, dams ce quotient ne sont 
pas distinctes en général. On a ti, ti, --- ti, = bj, bi bap si et seulement si les 
suites (71, 72,-.-,%p) et (j1,32,--- ,jp) se déduisent l’une de l’autre par permu- 
tation circulaire. Quotientons alors simultanément (R™)? et XP? par Vaction 
du groupe cyclique C, et restreignons-nous aux suites (i1,72,...,%p) telles que 
les 7, ne soient pas tous égaux. L’expression 


Sy tj, tig ++i Zi, IN Goes Nos Ae. 


(i1 iP peccdey) 


a alors un sens dans l’espace ((R™)?/C,)™ xo,, (X*?/Cp)™, avec m = 
(nP — n)/p. En passant la limite inductive suivant n, on en déduit bien 
une application 


p: Q(X) — Q(CP,(X)), 
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désignée aussi par “puissance réduite mod. p” (la relation entre cette applica- 
tion et celle déduite du théoréme de J. Lannes sera faite en 4.15, A la fin de ce 
paragraphe, grace 4 un argument de transfert). 


4.9. Contrairement 4 ce que nous avons vu en homologie et cohomologie ordi- 
naires (cf. 2.3 et 2.7), application » : Q(X) — Q(CP,(X)) ainsi construite 
se factorise par Q(ECp xc, X*?/BCy) := Q(EC} Ac, XP), o ECp désigne 
espace EC, U {oo} (L’espace ECT Ac, X”? sera noté simplement CP?(X): 
c’est le “produit cyclique homotopique” de X.) En effet, si >> tiz; € Q(X), 
on peut lui associer |’élément suivant de Q(ECT Te, AO) 


ye Cc Gy : tin ve se) lt, ti, vee ti, Li, A iba, (Nass IN cae : 
(i1,42,.--5¢p) 
Dans cette expression, c(t;, - ti,---ti,) désigne la classe de t;, - ti, a 
dans (R™)?/C, et le couple [t;, -ti.---ti,,2:1, AZig A--- A z;,| représente 
un élément de JBICn Ac, XP qui ne dépend que de la classe (i1, i2,.. wis): 


Puisque les c(t;, - ti, ---t;,) sont distincts, on a bien défini une application 
pg”: Q(X) — Q(CP#(X)) telle que le diagramme suivant commute 


Q(X) — Q(CP}(X)) 


| 


Q(X) — QCP,(X)) 


Comme nous l’avons vu en 2.3 et 2.7, l’espace OE (X) est lié au produit 
cyclique de X par un diagramme cocartésien 


BC, x X/BC, ——3 CPx) 


| | 


e ers 


Puisque le céne de application BC, x X/BC, — X est la suspension de 
BCZ AX, on a une cofibration homotopique 


CP?(X) — CP,(X) — S'A Behe 
La factorisation montrée plus haut montre donc que l’application composée 
Q(X) — Q(CP,(X)) — Q(S1.A BCt AX) 


est homotope 4 0, en contraste avec la situation homologique (cf. 2.7). En 
particulier, l’application composée 


Q(S") — L(S") — L(CP,(S")) — L(S' a BC+ A 8") 
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est homotope A 0: ceci implique que les opérations de Steenrod H4(X )— 
H%+"(X;Z/p) sont nulles sur Vimage de V’homomorphisme de Hurewicz 
n1(X) —> H4(X) sir > 0. Bien entendu, cette nullité des opérations de 
Steenrod sur l’image de l’homomorphisme de Hurewicz peut se démontrer de 
maniére élémentaire. Cependant, le caractére plus précis que nous avons donné 
pour cette annulation laisse espérer des “opérations de Steenrod secondaires” 


(ei 2210): 


4.10. Le choix d’un point base dans EC, (par exemple, l’image de |’élément 
neutre e de EC, qui est un groupe abélien) définit une application X“? —- 
CPCS), soit £1 A 22+: ALpt> (€,£1 A 22°-: A Zp). L’application composée 


XP —+ CP}(X) — CP,(X) 


est alors l’application quotient canonique. En particulier, cherchons l’image de 
la puissance p'*™* de « = ) > tix; dans Q(CP#(X)). Ona 


gg? a= » ti, tig bi, Tig NDig Ave A Xa, 3 


(41 ,12,---,tp) 


Son image dans Q(CP}(X)) est 


alge OOP ue Ss; ti, big aed eal Ly ASE Sh A Ti) 


(41 ,2,.--,;tp) 


ou la dernigre somme est indexée par des suites (i1,72,-.- ,ip), les ig n’étant 
pas tous égaux. Cet élément est homotope (en tant qu’application de Q(X) 
dans Q(CP; (X)) & la somme suivante: 


Sy ti, tip os ti, lies o ti, Vix I dos Noe A xi, 


(i1 ,42,---s2p) 


(utiliser la contractibilité de R~()), En choisissant un domaine fondamental 
de l’action de C, sur R~(?), on voit que cette expression s’écrit aussi comme 
p fois la somme suivante dans Q(CPh(X): 


c (ave ag -ti,) [tepees bi, Lay (A Bay NO? IK xi, 


(1 ,2,---:tp) 


(c(G, Gate, ) désignant la classe de tj, ti, ---h, dans le domaine fonda- 
mental), c’est-a-dire précisément la “puissance réduite” g”(x) de z dans 
Q(CP?(X)) qui a été définie en 4.9. 
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4.11. Comme en 4.6-7, considérons le cas particulier ou X est une sphere S%. 
Pour p impair, d’aprés ce qu’on a vu plus haut, la p-puissance réduite définit 
donc une application 


Q(S*) — Q(CP;(S*)) = Q(S"(BCp/Lap)) - 


Si p = 2, les calculs faits en 4.7 montrent que la puissance réduite définit aussi 
une application 


Q(S1) — Q(CP3(S1) = Q(S4(RPoo/RPy-1)) « 


La relation entre cette puissance réduite et la puissance p'*™* ordinaire sera 
faite en 4.13. 


4.12. Nous nous proposons de faire le lien entre les deux points de vue?? 
que nous venons de développer sur les opérations de Steenrod en cohomotopie, 
grace 4 une application de transfert analogue 4 celle définie en homologie et 
en cohomologie (cf. 1.1). Pour cela, considérons un groupe fini G opérant 
librement sur un espace pointé V8 (sauf au point base) et posons U = V/G. 
On peut alors définir une application “transfert” 


T:U—Q(V) 


de la maniére suivante (communication de J. Lannes). I] convient d’abord de 
plonger G dans un groupe symétrique 6, et V dans R& ) U Os R~(")+ de 
maniére équivariante (c’est possible car V est un CW-complexe de type fini). 
On a donc le diagramme commutatif 


yv—~.) Ro)+ 


a | 


UG — + R)+/G 


(G Cc G, opére sur R%(")+ par permutation des coordonnées). L’application 
T cherchée associe alors au point u de U 1’élément 


r(u)= S> Alw)o, 


a(v)=u 


ot A(v)-v € Re) xg, V". L’application 7 s’étend de maniére usuelle & OM): 
il suffit de poser 


/ (= tm ne yo ta A(va) Va - 


T(Yo j=29 


12 espace des points fixes homotopiques et quotient homotopique. 


13 On suppose ici que V est un CW-complexe de type fini. 
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Dans cette formule, l’expression t, -A(v) doit étre comprise comme un élément 
de R® x R™("), L’image du transfert appartient au sous-espace Q(V)° de 
Q(V) formé des éléments invariants par G (G opérant simultanément sur V 
et R~-(7)), Le transfert peut donc étre interprété aussi comme |’application 
composée 


QU = Q(V)* = OV)" . 


Sous cette derniére forme, compte tenu du lemme 4.4, il suffit de faire opérer 
G sur V. 
Une premiére application du transfert est le théoreme suivant. 


4.13 Théoréme. L’application puissance p*™* “normalisée” en cohomotopie, 
soit la composition 


Q(X) — Q(X?) —> Q(X"?/X) 


se factorise 4 travers Q(CP/(X)). De maniére précise, on a le diagramme 
commutatif 


Q(x) —— ax) —>  Q(x""/x) 
| Is 


Q(CPH(X)) Q(EC, x X*?/EC, x X) 


ou 7! désigne la projection Q(CP?(X)) —> Q(CP}(X)/BC, x X), composée 
par le transfert 


7: Q(CP*(X)/BC, x X) =Q(ECp xo, X’?/ECy xc, X) 
hee XP Omen 


Démonstration. Soit un plongement de X dans R®™. D’apres 4.9, 
application composée 7’ -o" associe Az = >) tjx; € Q(X) lexpression suivante 


D c (Ge: aoe sie) : ti, : (BBs S004 t;, (Za, ) (Zin) sisay Gi.) 
(41,42,--.,tp) 


[tis Seed tn5 Pty IS Bois A+: A2i,| 


out les ig ne sont pas tous égaux. Ce calcul est licite car Vaction de C, sur 
XP — X est libre. L’élément (7’ - p")(x) ainsi trouvé est canoniquement 
homotope (par une homotopie affine) A une somme analogue ot on remplace 
Vexpression ¢ (ti, ,tig,--+sti,) tir + tian ,tyu(a;, (zig) ++ w(24,) simplement 
par t;,t;,°-:ti,- Modulo X, nous pouvons aussi Supposer que les indices ig 
sont quelconques: on retrouve bien la puissance p*™*. 
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4.14 Corollaire. Si p est un nombre premier impair, on a un diagramme 
commutatif 


Q(S9(BCp/L,,p)) = Q(CP?(S4)) 
ve 
Q(S") | 
“ 
Q(S?2) 


ou la fléche \, représente la puissance p**™. Si p = 2, on a la factorisation 
suivante: 


O(S7(RP../ Rie) O(C PB, (S2)) 
7 
Q(S4) | 
“ 
OS) 


Dans les deux diagrammes, la fléche verticale Q(CP/(S1)) —+ Q(S?2) est la 
composée 


Q(CP;(S7)) — Q(CP,(S?)) — Q(S?*) 


ou la deuriéme application est induite par le “transfert” 7 : Q(CP,(S4%)) —> 
QS"). 

Démonstration. Sans restreindre la généralité, on peut quotienter CP,(S%) 
et SPY par S4, ce qui revient 4 ajouter un facteur Q(S4*1) & la formule (cf. 
1.10). Le transfert n’est défini en toute rigueur que pour des actions libres, 
sauf au point base, ce qui explique les quillemets dans l’énoncé ci-dessus. En 
ajoutant le facteur Q(S4*"), on se raméne au théoréme précédent avec X = S74. 


Faisons maintenant le lien avec le travail de J. Lannes mentionné en 4.6 et 
4.7. 


4.15 Théoreme. On a le diagramme commutatif suivant: 


OCC ON eRED) ace 


Is 


h+ 


Q(X) + Q(BCy x X//EC, x X)POr 


ou. g désigne la puissance p*™ équivariante définie en 4.5 et ou gt désigne 
Vapplication composée 


h 
Q(X) © Q(ECy xo, X"?/ECy xe, #) — Q(ECp xc, X"?/ECp xe, X) 
Q(EC, x X"P/EC, x X)? —+ Q(EC, x X’?/EC, x X)POr, 
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T désignant le transfert. 


Démonstration. I] suffit de répéter la démonstration du théoreme 4.13 en 
tenant compte du caractére équivariant de la puissance p!*™°. 


4.16. Analysons enfin g*(x° + 1), ob 2° = 372929 et z! = \ otha}. D’aprés 
4.9, nous pouvons écrire 

eee 
e(et+s')= >> ol aan) eee te act hae A Ane] 


(71,42 CBGcH ip) 
(€] .#91-+-. ep) 


avec €g = 0 0u 1. Avec les notations de 2.1, nous pouvons définir de meme 
une application y, : Q(X) x Q(X) — Q(EC, x X*?/EC,) par la formule 


suivante (ot les €, ne sont pas tous égaux): 


Oa ea) — Dy C (@ i ee) ier DRS Ra ean TA za : 


412 “ig? °* tp 241 22 tp? cay tp 
Dl Gagwostin)) 


Comme en 2.1, il est clair que "(2° + 2!) = g(x) + p(x!) + T(yp(2°, 2')), 
ou 


Oem Hole O(hOnxc, X°?/ BC) 


est l’application canonique. Puisque EC, est contractile, on peut remplacer le 
premier espace par Q(X%?). On en déduit le théoréme suivant, qui mesure le 
“degré d’additivité” de lopération de Steenrod a”. 


4.17 Théoreme. Soient x° et x! deux éléments de Q(X). On a alors la 
formule suivante 


p(2° + 21) = p(x) + *(2") + T(@p(2°, 2") 
ou T : Q(X”) —+ Q(EC, xc, X*?/BC,) est Vapplication canonique et ou, 


avec les notations de 2.1, p : Q(X) x Q(X) — Q(X?) est définie par 
Vexpression 


(Oy ocd lees Ey 4&2 Ep al BN ao Ep 
Pp(x x) = y o(t8,t2,... 12) aft Aoi A Na; . 


@,(t1,---;tp) 


En particulier, pour p = 2, pa est le cup-produit. 
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Annexe A. Cohomologie bivariante et isomorphisme de Thom '4 


A.1. Dans cette annexe nous rassemblons quelques résultats sur la cohomologie 
bivariante et l’isomorphisme de Thom que nous n’avons pas trouvés dans la 
littérature sous cette forme. Rappelons (cf. 2.8) que si X et Y sont deux 
CW-complexes pointés, le groupe HH(X,Y) est défini comme l’ensemble des 
classes d’homotopie d’applications pointées de Y dans L(X). Ce groupe HH 
vérifie des propriétés analogues & celles de la K K-théorie de Kasparov. En 
particulier, un accouplement bilinéaire 


HH(X,Y AA) x HH(AA X’',Y’) — HH(XAX',Y AY’) 
est défini de la maniére suivante. A partir de f’: Y’ —> L(AA X’), on déduit 
une application Y AY’ —+ YA L(AA X’) — L(Y AAA X’). L’application 
Y AA — L(X) et la précédente se composent en 


YAY’ — L(L(X) AX") — L(L(X A X’)) — L(X AX’). 


Elles définissent ainsi l’accouplement cherché. 


A.2. Cas particuliers (ici H, et H” désignent respectivement ’homologie et 
la cohomologie réduites). 


1A Soo y= sey’ — 52 On en dédint Vaccouplement 
Hn(X) x Hp(X’) — Hnip(X AX’). 

2. A=S°,X =S", X’ = S?. On obtient alors l’accouplement 
AY ex BY HP aay), 


Les fléches décrites en 1. et 2. représentent évidemment les cup-produits 
usuels en homologie et en cohomologie. 


3. X’ = S°, X = S$", Y’ =S?. On obtient alors le “slant-produit” 
H"(Y A A) x H,(A) — H”-?(Y). 
4. Enfin, un autre type de produit 
HH(X,Y) x HAY, Z) — HH(X NY, Z) 
FU ae aie came s’inspire en partie de l’article: “Formes 


topologiques non commutatives,” paru aux Annales Scientifiques de l’Ecole 
Normale Supérieure [2). 
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est défini comme la composition des fléches suivantes (la premiére étant 
induite par la diagonale de Y dans Y AY): 


HH(X,Y)xHH(Y, Z) —> HH(X,Y) x HH(Y AY, Z) 
= HH COONY, 2) - 


Si X = S” et Z = SP, on en déduit le “cap-produit:” 
ial (WOV SOO alee Oe 
Il est parfois commode de poser 


HHi(X,Y) = HH(X AS?,Y) pour j >0 
HHi(X,Y) =HH(X,S-3 AY) pour 7 <0. 


Les foncteurs (X,Y) + HH*(X,Y) sont alors des foncteurs homologiques 
en X et cohomologiques en Y respectivement. 


A.3. On peut interpréter de maniére légérement différente ces définitions de la 
maniére suivante. Fixons un entier n > 0 et considérons le bicomplexe (p,q) > 
9P:9(X, Y) = Mor(BP AY, L(X AB4"")), oi p est restreint aux entiers < n. Les 
différentielles de ce bicomplexe sont induites par les applications canoniques 
B™ —+ B"+1 obtenues par la composition évidente B’ —+ S" —> B™*. 
En particulier, le sous-complexe gq > 2%9(S°,Y) = Mor(Y, L(B?t")) est le 
complexe de de Rham des “formes topologiques non commutatives” considérées 
dans [2]. Le bicomplexe général s’écrit schématiquement ainsi 


Qr2 ae Qr-1.2 ns Qr-2,2 ee 


i i T 


Qn Qqr-1l = Qr-2.1 pen 


T i} i 


qQ7.o — Qr-1,0 a Qr-2,0 ey ee 
En fait, les lignes sont exactes car la suite 
0 — Mor(S" AY, Z) —> Mor(B” AY, Z) — Mor(S"~! AY, Z) —>0 


est exacte si Z est un groupe topologique contractile. La cohomologie de ce 
bicomplexe est donc celle du complexe obtenu en considérant le noyau des 
premiéres fléches horizontales, soit le complexe 


Mor(S” A Y, L(X A B*)) —+ Mor(S” AY, L(X A B?)) 
—+ Mor(S” AY, L(X A B*)) —-:- 


dont la cohomologie en degré r est isomorphe Bre so 2 ia) 
HH (Ge): 
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A.4. Venons en maintenant a l’isomorphisme de Thom que nous allons in- 
terpréter dans notre langage. Ici V désigne un fibré vectoriel orienté de rang n 
et de base X compacte (cette hypothése n’est 14 que pour simplifier le raison- 
nement; il est facile de généraliser). Soit B(V) (resp. S(V)) le fibré en boules 
(resp. en sphéres) de V pour une métrique quelconque sur V. En particulier, 
V est homéomorphe 4 V’ = B(V) — S(V). 

La classe de Thom de V peut étre représentée par une application continue 
B(V) — L(S") égale 4 0 sur S(V), grace & un argument standard du type 
Mayer-Vietoris. Par ailleurs, l’application diagonale de V’ dans V’ x V’ induit 
une application propre de V’ dans B(V) x V’. En considérant les compactifiés 
d’Alexandrof, on en déduit les applications continues suivantes (ot 7(V) = 
B(V)/S(V) désigne l’espace de Thom de V, interprété comme compactifié 
d’Alexandrof de V’): 


7(V) —> B(V)* Ar(V) — Xt ar(V) — Xt AL(S"). 
A.5 Theoreme (Thom). L’application composée précédente 
T(V) — Xt AL(S") 
indutt les isomorphismes suivants pour tout espace pointé Y: 
HH~"(Y, Xt) — HH(Y,7(V)) et HH(r(V),Y) — HH" (XY). 
En particulier, pour Y = S?*", on en déduit des isomorphismes 
H?(X*+) > HP*"(r(V)) et Hp(r(V)) — Hp-n(Xt). 


Démonstration. Elle résulte d’un argument standard de Mayer—Vietoris (on 
se raméne au cas ou V est trivial). 


Annexe B. Transfert et espaces lenticulaires. 


Le but de cette annexe est de rassembler quelques résultats (sans doute bien 
connus) sur les espaces lenticulaires qui facilitent la compréhension des §1 et 2 
de cet article. Nous en donnons ici des démonstrations completes. 


B.1 Théoréme. Soit C"*! un espace vectoriel compleze euclidien muni d’une 
action linéaire du groupe G = Cy, p premier impair. On suppose que cette 
représentation est somme directe de représentations irréductibles non triviales. 
Soit S?”* la sphére de C"+1. Alors, si R = Z, Vapplication de transfert 
détaillée dans le §1, soit 


a L(S2" 11 /G) = (gen yne 


est une équivalence d’homotopie. 
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Démonstration. Nous allons montrer que |’application a induit un isomor- 
phisme sur tous les groupes d’homotopie 7;. Ceux-ci sont égaux a 0 pour z 
pair, a Z/p pour i impair 4 2n+ 1 et a Z pour i= 2n+1. Pour le facteur Z, 
remarquons d’abord qu’il est déterminé pour le premier groupe par le transfert 
L(S2"+1/G) —+ L(S?"*1) qui induit un isomorphisme sur les groupes T2n+1 
(cf. 1.11 et 1.13). Par ailleurs, soit 


478 (GO) I Se ee (Sea eel 


ot M comprend p facteurs L(S2"+1), application définie par 


=i 
1G) (eee ae 


o étant le générateur du groupe cyclique C,. Alors + est équivariante pour 
action de C, sur M consistant & permuter circulairement les facteurs. Soit 
N= Ton+1(M) ae aP. 

Puisque H‘(G; N) = 0 pour i > 0 et que H°(G; N) = Z, y induit une ap- 
plication L(S2"+1)h¢ —. (M)*¢ = L(S?"+") telle que le diagramme suivant 
commute (¢ étant le transfert): 


L(S2 7G) ge aS 


Saw 


(Seo ) 


L’application a induit donc un isomorphisme Toh (S2"" {G)) 
Tani (L(S2"+1)"@) = Z. En particulier, sin = 0, on a Si (E(S5/G)) 
voy GES . 

Nous allons démontrer le théoréme général par récurrence sur n en sup- 
posant done L(S$2"~1/G) ~ L(S27-1)G_ Pour cela, écrivons le diagramme 
commutatif 


Re 


L(S?"-1/G) eee L(S?"+1/G) pe sae L(S2"+1) 


| | 


TEC ratemee Whies LS?" ee t ; LSE) 


En appliquant le foncteur ™, 0 <7 < 2n—1, on obtient le diagramme suivant 


(se GC) ee (5G) 


| | 


ni(L(S29-1)h2) ass, Pee (oot) Pe) 
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Pour 2 pair, tous les groupes sont nuls. Pour 7 impair, il est bien connu que 
la premiére fléche horizontale est un isomorphisme si i 4 2n — 1 et que c’est 
aussi un isomorphisme pour i = 2n — 1 si on quotiente le premier groupe par 
p. Pour démontrer le théoréme, il suffit donc de montrer la méme propriété 
pour la deuxiéme fléche horizontale. 

Pour accomplir ce programme, considérons la fibration vectorielle 


V = g2nti _ gl _, g2n-1 


Plus précisément, écrivons C”t! = Lo@L1@---@L,n (somme de représentations 
irréductibles); S?"+1 est la sphére de C"*+1; $1 est vue comme la sphére de Ln. 
La structure vectorielle est déduite du diagramme cartésien classique suivant: 


g2ntl fax si een g2n-1 


| | 
GR (e's) Ee oe 


avec 6([%0,21,---,2n]) = [0,71,.--,n—1], ot la fibre au-dessus du “point” 


L sidentifie 4 Hom(L,L,). Si on munit ce fibré vectoriel d’une métrique 
invariante, l’espace de Thom V+ s’identifie au quotient B(V)/S(V) du fibré en 
boules B(V) par le fibré en sphéres S(V), soit aussi S?°+1/S1. En quotientant 
par l’action de G, on a encore une fibration vectorielle 


W= (es = S')/G a Siam C7 


dont l’espace de Thom est ($?"+1/S1)/G. Soit 7 la “classe de Thom” de cette 
fibration, vue comme application Wt —+ L(S). Elle induit la classe de Thom 
de V grace a la composition Vt —> W+ — L(S?) (car G opere librement sur 
la base). L’isomorphisme de Thom de V peut étre ainsi réalisé au niveau du 
produit symétrique infini (cf. la fin de l’annexe précédente) par une application 
équivariante T(V) — (S*"—1)* A L(S?) qui induit une fléche équivariante 


0? (L(S?"*7 /S1) —+ L(S2-1) . 
On en déduit un isomorphisme 
ma(L (S23 )) —+ m2 L(S2"41)2) 
et un diagramme commutatif 


H(S*/G) —> B(s™4/G@) —. Hy o(5?"1/¢) 


| | | 


a(S ee) eat ig (Se) =e og: G(s Ee) 
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L’application composée H;($?"-1/G) —+ Hj_2(S?"~!/G) est un isomor- 
phisme si 2 < i < 2n —1 et est aussi un isomorphisme pour 7 = 2n — 1 si 
on quotiente le premier groupe par p. II en résulte que la méme propriété vaut 
pour l’application composée 1;(L(S2"-!)?©) —+ m_2(L(S?"-1)?¢). Done 
a: H,(S?"-1/G) — 1;(L(S?"-1)?°) est aussi un isomorphisme. 


B.2. Théoréme. Soit R™ = C” @R%, ot CO” (resp. R4) est somme de 
représentations irréductibles du groupe cyclique Cp (resp. est un Cp-module 
trivial). Alors application canonique 


Cel Ge eG) ge) 


induit un isomorphisme sur les groupes d’homotopie 7 pour i > q. St on 
remplace le foncteur L par £ @ Z/p, la méme conclusion vaut pour t > q. 


Démonstration. Il est plus commode de considérer S™ comme le com- 
pactifié d’Alexandrof de R™ qui s’identifie 4 la suspension (non réduite) de 
S™-1 en tant que G-espace. Nous pouvons ainsi écrire iS a! 
et S27+9/G = (S?"/G) A S41. De manieére générale, notons S'Y la suspen- 
sion non réduite d’un espace Y. On a alors les équivalences d’homotopie 
suivantes O(L(S2"+9/G)) = Q(L(S'(S*t49-1/G)) = WG ete /G). et 
G22 FS ML (SSA tt 1) PS) L(S2"+4-1)hG_ De méme, nous 
avons 09+1(L(52"+4/G)) = 0941(L(S/941(S2"-1/G))) = L(S2"s 1G) et 
Or (S22 aye) Oat (S't 1 (G28) Pe) L(S?"-1)G_ La premiére 
partie du théoréme se déduit ainsi du théoreme précédent, en appliquant le 
foncteur 7; aux isomorphismes ci-dessus pour 7 > q. La deuxiéme partie s’en 
déduit aussitét en écrivant les suites exactes d’homotopie reliant 7;(L(X)) et 


n;(L(X)) @ Z/p. 


B.3. Nous nous proposons d’étendre les résultats précédents en homologie 
mod. 2, c’est-a-dire en supposant R = Z/2. Pour éviter toute confusion, nous 
noterons ici L le foncteur L pour R = Z/2, en conservant la notation premiere 
L dans le cas o1 R= Z. Si X est un G-espace (avec G = C2), nous noterons 
Xo le quotient de X par le sous-espace des points fixes X©. Il est clair que le 
transfert usuel induit en fait une application (notée encore t) 


L(Xo/G) — L(X)*° 


(car le transfert usuel s’annule sur THX S) Ch 13), 


B.4 Théoréme. Soit R" = R™ @ R? un espace euclidien ot le groupe G= 
Cy opére par involution antipodique sur R”™ et par Videntité sur R4% (avec 
q> 1). Le groupe G opére donc sur la sphére correspondante S” (vue comme 
compactifiée d’Alexandrof de R”), l’espace des points fixes (S")° s’identifiant 
4 S47. On pose St = S"/(S")%. Alors l’application de transfert sur la sphere, 
soit 


a: L(S¢/C2) — RSE 
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L=L®Z/2) induit un isomorphisme sur les groupes d’homotopie 7; pour 


1>q. 


Démonstration. Elle se divise en deux parties: 


Il, 


Supposons d’abord i > q+ 2. Nous avons alors le diagramme commutatif 
suivant de fibrations (avec G = C2) 


OCS 22/ 6) = (Sts ee (Gaye) 


| | | 


O(L(S™+9)h) ae L(S™+1)hG _ 2, L(S™+4)hG 


En écrivant les deux suites d’homotopie correspondantes, ainsi qu’en ap- 
pliquant les théorémes B.1 et B.2 (adaptés au cas réel et appliqués en 
homologie mod. 2), on en déduit que 7;(L(S"*9/G)) = 7;(L(S™+9)r@) 
pour 7 > q+ 2. Par ailleurs, l’espace des points fixes de S™+9 par l’action 
de G s’identifie 4 S? et l’application SY —+ S™+4/G est homotopiquement 
triviale. Par conséquent, L(S§/G) ~ L(S™+47/G) @L(S4*1) et le théoréme 
est bien démontré pour 7 > q+ 2. 


. Supposons maintenant que i soit égal A g+1. Dans ce cas, |’homomor- 


phisme 
eget: Z/2% %q41(L(S3/G)) —> ma s1(L(S™*4)"°) ws Z/2 


est égal 4 0 ou est un isomorphisme pour tous g et n. En effet, i] suffit 
de répéter ’argument exposé dans la démonstration du théoréme B.1 (en 
remplacant l’homologie entiére par l’homologie mod. 2). La non-trivialité 
de a,+1 résultera donc d’un exemple approprié. De maniére plus précise, 
considérons le transfert introduit en 1.3 (avec p = 2): 


4: AE(S%) — L(S?4yhe | 


On doit montrer que 7 induit un isomorphisme sur les groupes d’homotopie 
™q+1 = Z/2. Pour cela, considérons la premiére opération cohomologique 
K(Z/2,q) —> K(Z/2,q+1) induite par la puissance 2'™e et qui résulte de 
la décomposition de L(S?4)"© en produit d’espaces d’Filenberg—Mac Lane 
(cf. annexe C). Cette opération se factorise A travers Ad (S%) par la 
puissance réduite (cf. 1.10). Il suffit done de montrer que l’application 
K(Z/2,q) —+ K(Z/2,q+1) n’est pas nulle: en fait c’est V?homomorphisme 
de Bockstein d’aprés [3]. Mais cet homomorphisme n’est pas trivial (con- 
sidérer des produits d’espaces projectifs par exemple). Ceci achéve la 
démonstration du théoréme B.4. 
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Annexe C. Type d’homotopie des groupes abéliens simpliciaux. 


C.1. Il est bien connu qu’un groupe abélien topologique X a le type 
d’homotopie d’un produit d’espaces d’Eilenberg—Mac Lane (cf. [8]). Comme 
me l’a signalé R. Jardine (communication privée), cette décomposition en pro- 
duit n’est pas canonique. Cependant, nous allons voir que cette décomposition 
peut étre rendue “canonique” si 7(X) est un groupe abélien de type fini, nul 
si n est assez grand, avec les choix supplementaires suivants (ceci est un cas 
particulier de résultats plus généraux de R. Jardine a paraitre): 

1) Choix des générateurs des facteurs cycliques libres de 7,(X). 


2) Pour chaque facteur cyclique Z/r de 7,(X), choix d’un élément de 
Tn—1(X;Z/r) dont image dans 7,(X) par Phomomorphisme canonique 


Tn—1(X;Z/r) — t(X) 


est un générateur de ce facteur cyclique. 


Plus précisément, soient G?, les facteurs cycliques de 7,(X) et soient a 
ies générateurs correspondants. Nous définissons une application de l’espace 
d’Eilenberg—Mac Lane K(G*%,,n) dans X de la maniére suivante. 


Premier cas. Gi, = Z. Choisissons comme modéle de K (Gin) le produit 
symétrique!® infini SP°(S”) de la sphére S” (cf. 2.2). Le générateur zt, définit 
une application essentielle a3, : S" —> X qui s’étend en exe SPEEA) <8 
par la formule 


Co 
G2 (2, - Try.) = de ahr) : 
r=1 


Bien entendu, la structure de groupe abélen de X intervient ici de maniére 
essentiele. 


Deuriéme cas. Gi, & Z/rZ. On procéde de maniere analogue en choisis- 
sant comme modéle de K(Gi,,n) le groupe abélien SP™(S")/rSP™(S”). 
Cependant, une difficulté technique apparait ici: les 6%, multipliés par r, ne 
sont pas égaux 4 0, mais homotopes a 0. Choisissons alors un relévement 
&, de a‘, dans le groupe d’homotopie T,—1(X;Z/r) par Phomomorhpisme 
canonique T—-1(X;Z/r) —> T(X). Alors @;, définit une classe d’homotopie 
d’application de l’espace de Moore S7 dans X (S™ est le céne de l’application 
de degré r de S” dans S”). Le produit symétrique infini de S? ayant le type 
d’homotopie de SP*(S")/rSP™(S”) = G2), on peut définir 6%, comme 


dans le premier cas, grace au choix de ce relevé Qj,. 


15 On peut aussi choisir le groupe abélien libre de base un modéle simplicial 
de S”. 
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En faisant la somme de tous ces (%,, on en déduit une application du 
produit des K(Gt,,n) dans X qui est une équivalence d’homotopie d’aprés le 
théoreéme de Whitehead. 


C.2. Supposons que X soit un R-module simplicial, les groupes d’homotopie 
étant des R-modules libres. Dans ce cas, le choix d’une base des groupes 
d@homotopie définit des applications'® L(S") —+ X. On en déduit directe- 
ment une application d’un produit des L(S") dans X qui est une équivalence 
d’homotopie, sans le détour des espaces de Moore. 


C.3. Supposons maintenant que R = Z et considérons un groupe abélien sim- 
plicial lzbre homotopiquement équivalent 4 un espace d’Eilenberg—Mac Lane 
K(Z/r,n). Alors X/r = X ® Z/r est homotopiquement équivalent a 
K(Z/r,n) x K(Z/r,n + 1), comme on le voit en écrivant la suite exacte 
d’homotopie associée 4 la suite exacte de groupes simpliciaux 


0X — X — X/r — 0. 


Le résultat suivant est utilisé en 2.11. 


C.4 Proposition. Avec les hypothéses précédentes, l’application canonique 
X = K(Z/r,n) — X/r ® K(Z/r,n) x K(Z/r,n +1) 


sidentifie au produit de l’application identique par l’homomorphisme de Bock- 
stein. 


Démonstration. D’aprés ce qui précéde, il existe une application L(S") —> 
X qui est une équivalence d’homotopie, S” étant un espace de Moore. Sans 
restreindre la généralité, on peut donc supposer que X est un espace de Moore. 
Par ailleurs, on peut écrire le diagramme commutatif 4 homotopie prés suivant 
(f étant une application de degré r): 


ep (ses) 


lee =| 


0 — > Ls") — > L(s") —— L(S")/r —+0 


Dans ce diagramme, les fléches horizontales sont des fibrations homotopiques 
et application “bord” L(S?) —+ Q7'(L(S")) & L(S"+1) (suite de Puppe) se 
factorise donc en L(S?) ~» L(S")/r —+ Q-}(L(S")) & L(S"*?), ¢’est-a-dire 
essentiellement par l’homomorphisme de Bockstein. La proposition s’en déduit 
en réduisant Q-!(L(S")) mod. r. 


16 L(S”) désignant le R-module simplicial libre de base S”. 
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Characteristic Classes of Singularity Theory 


M.E.Kazarian! 


1. Introduction 


Theorems of the global singularity theory express topological invariants 
in terms of singularities of some geometrical objects: vector bundle map- 
pings, the Thom-Boardman singularities of smooth mappings, Lagrangian 
and Legendrian singularities, RC-singularities ect. (cf. [P], [RJ, [L], [HL2], 
[AGLV], [V] ). A classical example is Hopf’s theorem relating the singular- 
ities of vector fields to the Euler characteristic. 

A lot of problems in this theory can be formulated as follows. Consider 
a smooth locally trivial bundle 7: E — M with fiber V and structure group 
G and a generic section s: M — E. A singularity class is any G-invariant 
subset in V. For a given singularity class =, the condition that the point 
s(b) € 7—1(b) = V, bE M, lies in © defines invariantly the set Se MM 
of points at which the section s has singularity =. The problem is to 
determine under what condition the intersection index with =(M) defines 
a cohomology class on M and to find an expression for this class. 

A general approach was suggested by V. Vassiliev. In his book [V] a 
cochain complex is constructed generators which correspond to singularity 
classes and the coboundary operator is given by adjacencies of singularities 
of neighbor codimensions. For any cohomology class = of this universal 
complex the intersection with 2(M) is well defined characteristic class of 
the given G-bundle over M. 

The homomorphism of the Vassiliev complex to the ring of characteris- 
tic classes of G-bundles is neither injective nor surjective. This means that 
some classes of the Vassiliev complex can give trivial cohomology classes 
on M and some characteristic classes of G-bundles cannot be expressed as 
classes dual to singularity classes. In addition, Vassiliev’s complex does 
not take into account that the restriction of the bundle to =(M) defines 
characteristic classes on =(M) itself and this can give new ‘derived’ classes 
on M. In this paper we show that using these ‘derived’ classes one can 
express all the characteristic classes of G-bundles. 

Formally, we construct a cohomological spectral sequence converging to 
the cohomology group of characteristic classes of G-bundles. This spectral 


1 Research of the author supported by the grants of International Science Foundation 
(grant Nr.MSD000) and of the Russian Foundation for Basic Research (Project Nr. 94- 
01—01203) 


326 M. E. Kazarian 


sequence is given by the classification of orbits of the action of G on V and 
described in terms of this classification. Corresponding to the singularity 
class = subgroup of E}’* is isomorphic to the cohomology group of some 
group Gg associated to this class =, namely, Gz is the stationary group 
of any representative f of =. The differential 6, is given by adjacencies of 
singularity classes of neighbor codimentions and higher differentials 6, are 
described by adjacencies of singularity classes whose codimension differs 
by r. Though the spectral sequence depends on the classification chosen, 
its terms beginning with the second one already do not depend on this 
classification. 

There is a natural isomorphism between the universal Vassiliev complex 
and the row E; > in the first term of the characteristic spectral sequence. 
Correspondingly, the cohomology of the complex is isomorphic to the row 
ES © of the second term. Thus, the Vassiliev complex which was originally 
defined with the help of an abstract algebraic-geometrical construction here 
appears as a result of calculations in the spectral sequence. 

In the previous paper [K2] we applied this approach to the classification 
of germs of function up to stable right equivalence and V-equivalence which 
give the characteristic classes of Lagrangian and Legendrian immersions 
respectively. In this paper we consider an application of the characteristic 
spectral sequence to the study of the classification of germs of functions up 
to (non-stable) right equivalence. 

Consider a structure of smooth fiber bundle on a manifold M (or, more 
general, a smooth foliation). Let f: M — R be any generic function. Then 
the singularities of the restriction of f to the fibers can form cycles on 
M. In this paper we prove that if the cohomology class dual to a cycle of 
singularities of some non-Morse type is correctly defined it is always trivial. 
In particular, this unexpected result show that the cohomology classes of 
the Vassiliev complex for R-classification of functions calculated in [AGLV] 
give always trivial cohomology classes on M. 


2. Example: characteristic Gysin exact sequence 


Let :E — M be an n-dimensional vector bundle over a compact 
manifold M and s:M — E be a generic section. Zeros of this section form 
a smooth submanifold F C M of codimension n. It is well known that the 
mod2-cohomology class given by the intersection index with F coincides 
with w,(m), the n-th Stiefel- Whitney class of the vector bundle zx. 

This assertion has the following interpretation. In what follows by the 
cohomology we mean the cohomology with coefficients in Z2. The inclusion 
a: F' — M induces the Gysin homomorphism 


14: H*""(F) —+ H*(M) 
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defined as a composition of the following two homomorphisms 
ee (MM eM), 


where the first arrow is the composition of the Thom isomorphism of the 
normal bundle of F' with the excision isomorphism and the second one is 
induced by the inclusion of topological pairs (M,0) C (M,M\ FF). An 
equivalent definition of i, is the composition of the Poincaré duality in 
F, the usual homomorphism 7, of the homology groups and the Poincaré 
duality in M. 

The assertion above stays that the Gysin homomorphism takes the 
fundamental class of F' to w,(7). 

Now, let 7 be any characteristic class of the restriction of the bundle 
x to F, say, some Stiefel-Whitney class. Then i,() is also a characteristic 
class i.e. it does not depend on the section S but on the bundle 7 only. In 
other words, a homomorphism 7, exists which makes the following diagram 
commutative 

H*-"(BO(n)) +H" (BO(n)) 
| et 
H*-"(F) —> H*(M) 

2.1. Lemma. The homomorphism Yn is given by the multiplication by 

w, € H"(BO(n)). 


Proof. The lemma follows from the so called projection formula 
i, (t*(a)b) = ai,(b) 


applied to b= 1 € H°(F) anda=ne H*(M ) any characteristic class. 

The lemma implies that Yn is injective. The cokernel of yn, is isomorphic 
to the ring of polynomials of the n—1 first Stiefel-Whitney classes that is to 
the ring of characteristic classes of (n — 1)-dimensional vector bundles. This 
gives the following splitting exact sequence which we call the characteristic 
Gysin exact sequence 


_.. SS H*-(BO(n)) 3 H(BO(n)) > H(BO(n-1)) 5, HF-"+1BO(n)) 33... 


The Gysin homomorphism 7, takes part in the Gysin exact long se- 
quence 


= HE" (F) 45 B*(M) > H*(M\ F) > BF) Ler 


which is nothing but the exact sequence for the pair (M, M \ F) with the 
cohomology groups of the pair (M,M \ F) exchanged by the isomorphic to 
them cohomology groups of F with shift of dimensions by n. 
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The discussion above leads to the following result which generalizes the 
assertion stated at the beginning of this section. 


2.2. Theorem. There is a natural functorial homomorphism between 
the characteristic Gysin exact sequence and the Gysin sequence for the sub- 
manifold F Cc M. 


A smooth mapping of manifolds y: N — M induces an n-dimensional 
vector bundle y*z over N, its section y*s and the homomorphism between 
the Gysin sequences for the pairs F C M and y~1(F) C N. The functori- 
ality mentioned in the theorem means the functoriality with respect to this 
homomorphism of the Gysin sequences. 

This theorem is a particular case of more general Theorem 3.7. One can 
give also its direct proof which uses the standard arguments of the theory 
of characteristic classes. 

According to this theorem the calculation of the characteristic classes 
for the bundle a can be reduced in some sense to that of the restriction of 
m to F and to that of the (n — 1)-dimensional factor bundle over M \ F of 
the bundle 7 over the line bundle generated by the section s. 


The same result holds for the Euler-Pontrjagin characteristic classes 
in the orientable case and for the Chern classes in the complex case. It 
would be interesting to interpret this results in terms of geometric residue 
theorems for the Chern-Weil theory [HL1,HL2]. 


3. Characteristic spectral sequence of G-bundles 


In this section by the cohomology we mean the cohomology group with 
the coefficients in a fixed abelian group A, say A = Z, Zz or Q. 

Let a Lie group G acts algebraically on a contractible algebraic manifold 
V. The linear representations theory of Lie groups give rise to a plenty of 
examples of such actions. 

Consider a (G,V)-bundle 7: F — M ie. a locally trivial bundle with 
a fiber diffeomorphic to V and structure group G. 

Let = C V be one of the orbits of the action of G on V. For 
ren (bh) 2V,bEM , the condition that x belongs to the orbit = is 
invariantly defined. This gives the classification of points of the bundle 
space EF in accordance with the classification of the orbits of the action G 
on V. Denote =(E) C E the set of points corresponding to the orbit =. 

For V is contractible the bundle a has a global smooth section s: M <3 
E and all its sections are homotopic to each other. Any section defines also 
the classification of points on the base M. Denote E(M) = s~'S(8). 


3.1. Lemma. Suppose s is generic. Then =(M) is a smooth subman- 
ifold the codimension of which is equal to the codimension Of =n V wie 
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structure group of the restriction m|=(m) of the bundle x to =(M ) can be 
reduced to Gz, where Gz is the stabilizer of any point v € 3(M). 


Proof. The first assertion of the lemma follows from the transversality 
theorem. To prove the second one it is enough to observe that the structure 
group of 7|=(m) is reduced to Gz if we choose only such trivializations of 
m|=(m) over domains in =(M) in which the section s takes the form s(b) = v, 
b € =(M), where v € & is some fixed point. 

A smooth mapping y: N > M induces a (G, V)-bundle on the manifold 
N and the homomorphism 


y*: H*(M) — H"*(N). 


The characteristic class associates with every G-bundle over a manifold 
M a uniquely defined cohomology class in H*(M) which is invariant under 
homomorphisms y* above. 

One could hope that the characteristic classes of the given bundle 7 can 
be expressed in terms of characteristic classes of the smaller groups Gz on 
the smaller manifolds =(M). This idea leads to the characteristic spectral 
sequence which we describe now. 

To avoid difficulties in the case when V contains infinitely many orbits 
we introduce the following notion. 


3.2. Definition [V]. A cellular G-classification on a manifold V is any 
locally finite partition of it into disjoint sets (called classes) satisfying the 
following properties. 

1. All the classes are semialgebraic sets and also smooth manifolds 
(without boundary) and all adjacencies of classes satisfy the Whitney reg- 
ularity condition. 

2. G-equivalent points lie in one class. 

3. For any two connected components L,, Lz of one class there are 
G-equivalent points f; € L, fo € Le. 

4. For each class the codimension of the orbits containing in it is 
constant. 

5. The subset of =x G consisting of all possible pairs of the form (point 
of =, element of its stationary subgroup) together with the restriction of 
the projection = x G — & to this space is smooth fiber bundle over © and 
the quotient space of the class = with respect to G-equivalence is smooth 
and contractible. 

The existence of a cellular classifications is proved in [V]. For any two 
such classifications there is the third one which can be obtained as a sub- 
partition of the two given ones. 

3.3. For a given classification consider the filtration on V given by open 


subsets 
Fo C Fi Ge ae 
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where Ff; is the union of all classes the codimension of which is not greater 
than 7. This induces filtrations on F and M 


Fo(£) CAE) c--- cE, Fo(M) C Fi(M) c--- cM, 


where ¥;(£) and F;(M) are the union of all =(F) (resp. =(M)) with 
= C F;. With this filtration we associate cohomological spectral sequences 
EP@(E) and EP%(M) converging to the cohomology group H*(E) ~ 
H*(M). 

To justify the spectral sequence E?:4(M) we need to impose the gen- 
erality condition to the section s. The initial term of this spectral se- 
quence E?’*(M) is isomorphic to the cohomology of the smooth manifold 
Fp(M) \ Fp-1(M) of codimension p with the coefficient system orienting 
the normal bundle of this submanifold. 


3.4. Remark. It is more natural to consider this filtration by open 
sets rather than, say, the filtration formed by the complements of F; (M), 
cf. [K2]. Moreover, while from the point of view of, say, the theory of 
CW-complexes more complicated spaces are obtained from simpler ones by 
gluing cells of increasing dimensions from the point of view of the singularity 
theory more complicated spaces are obtained from simpler ones by cutting 
off smooth submanifolds of increasing codimensions. It would be interesting 
to build the general theory realizing such approach. As an example, we 
give the following construction of the classifying space for a Lie group GC 
GL(n,R). Consider a natural action of G on R” and the product action on 
R” @ RN = R"@...@R” (N times). Let Wy C R° @RN be the union 
of all free orbits. Then the codimension of the complement R” @ RY \Wn 
increases infinitely with N and Wy has zero homotopy groups for small 
relative to N dimensions. Thus, for any d there is a number N (d) such that 
the cohomology groups of the orbit space By = Wy /G are isomorphic to 
the cohomology group of the classifying space BG in all dimensions smaller 
than d. 


The filtration on M is induced by the section s from the filtration on 
E, F;(M) = s~!(F,(E)). Therefore s induces a homomorphism between 
the spectral sequences 


s*: Ey" (E) — Et" (M) 


In the same way the mapping y: N — M inducing a (G, V)-bundle on 
N induces also the homomorphism 


ENCES Te) TB 


where we denote y* E the total space of the (G,V)-bundle y*z over N. 
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3.5. Definition. The characteristic spectral sequence E*** is the induc- 
tive limit of the spectral sequences E*-*(E) over all (G.V)-bundles and over 
homomorphisms y* above between them. 


3.6. Proposition. The characteristic spectral sequence converges to the 
group H*(BG) of the cohomology characteristic classes of G-bundles. 


This follows from the fact that the group of characteristic classes can 
be defined as the inductive limit of the groups H*(M) over all G-bundles 
and homomorphisms y* between this groups, and from the properties of 
inductive limit. 

The following proposition is the direct corollary of the definition. 


3.7. Theorem. For a given (G,V)-bundle on M and its section s there 
is a natural homomorphism 


K*: B* — ES*(M) 
such that the limit homomorphism of the cohomology 
K*: H*(BG) — H*(M) 


is the characteristic homomorphism which evaluates a characteristic class 
on the given G-bundle over M. 


3.8. Example. The case of arbitrary n-dimensional vector bundles 
corresponds to the natural action of the group G = GL(n,R) on V = 
R”. The space V consists of two orbits, the origin and its complement. 
Hence, the considered spectral sequence have two nontrivial columns only. 
A spectral sequence with two nontrivial columns may be reduced to an exact 
long sequence which is in our case nothing but the Gysin exact sequence 
discussed in Sect.2. Therefore Theorem 3.7 takes form of Theorem 2.2. 


The following explicit construction can be taken as another definition 
of the characteristic spectral sequence. Let the principal G-bundle Wy — 
By be as in 3.4. Consider the associated with it (G, V)-bundle 7: En > 
By, Ey = Wn Xc V. Then the spectral sequence Ey" (En) converges to 
H* (En) & (Bn: 

3.9. Theorem. If N > N(d) then the homomorphism 

K*: EPI —+ EP4(Ey) 


is an isomorphism for p+q<d and all. 


Proof. A given (G, V)-bundle E > M over a smooth manifold M can 
be induced from the bundle 7: Ey — By by asmooth mapping ¢: M— Bn 
for N large enough. This mapping induces a homomorphism 


ot: BP"(Ey) —+ E(B) 
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which shows that «* in Theorem 3.9 is surjective. As y is uniquely defined 
up to a homotopy the homomorphism y* does not depend on the choice 
of the mapping y which shows that the epimorphism «* is, in fact, an 
isomorphism. 

The following theorem describes explicitly the term E}”* of the charac- 
teristic spectral sequence. Recall that if a class = Cc V consists of the only 
orbit the structure group of the restriction of a (G,V)-bundle over M to 
=(M) can be reduced to Gs. The same holds in the general case because 
the property 5) of Definition 3.2 implies that the stationary subgroups of 
all the points of the same class are isomorphic to each other (we denote it 
again by Gg). Hence, the given Gs-bundle over =(M) is induced by some 
continuos mapping ke:=(M) — BGs defined uniquely up to homotopy. 

If we realize the group Gz as the stationary subgroup of an element 
v € & Cc V then Gz acts on the linear space T,V/T,=. Some of the 
elements of Ge do not preserve the orientation of this space. Denote O= 
the coefficient system (a locally isomorphic to the abelian group A sheaf) 
on BGz which orients the vector bundle over BGs associated with the 
given action of Ge on T,V/T,5. Then the coefficient system Oz(m) on 
=(M) coorienting the normal bundle to =(M) C M is isomorphic to the 
coefficient system induced from Oz by the mapping ks and we get an 
unambiguously defined homomorphism 


ki: H*(BGz, Os) —> H*(&(M), Oza). 


3.10. Theorem. The term E>” of the characteristic spectral sequence 
ts isomorphic to the direct sum of summands corresponding to the classes 
of codimension p of the chosen classification of points on V. The direct 
summand corresponding to the class = is isomorphic to the group of the 
twisted characteristic classes H*(BGs,Oz) of Gz-bundles. The character- 
istic homomorphism 


KEP" = € H*(BGz,Os) — EP*(M)= @) H*(E(M), Osan) 
codim = codim ==p 
is the direct sum of the homomorphisms Ke. 


3.11. Lemma. Let Ey and (G,V)-bundle over Ey be as in Theo- 
rem 3.8. Then the space =(En) is isomorphic to the factorspace of the 


space with trivial N first homotopy groups over a free action of the group 
Gz. 


Theorem 3.10 is followed by this lemma and the description of the term 
Ey’*(M) given in 3.3. 


Proof of Lemma 3.11. If = consists of the only orbit the lemma is 
evident, under the isomorphism Ey = V xg Wy we have 


(En) = Gu XG Wn = {uv} XGs Wn => {v} x (Wn /Gs). 


(1) 
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In the general case it follows from the property 5) of Definition 3.2 that 
=(En) is the total space of the bundle with the topologically trivial base 
and the fiber as in the conclusion of the lemma. 

The differential 6,: E77? > E?*4 is given by adjacencies of classes of 
neighbor codimensions (see [K2] for its explicit description). The higher 
differentials 6, are given by adjacencies of classes the codimension of which 
differs by r. 

Formally, our spectral sequence depends on the chosen classification on 
the space V. In fact, for r > 2 its terms already do not depend on the 
classification. 


3.12. Theorem. For r > 2 terms E?4 of the characteristic spectral 
sequence are naturally isomorphic to each other for different cellular clas- 
sifications on V (with fired p, q, r). 


The proof word by word repeats the proof of the corresponding theorem 
in [K2]. 


4. Vassiliev universal complex 
and the characteristic spectral sequence 


Let 7: E — M be a (G,V)-bundle and s: M — E its generic section. 
One possible way to define a characteristic class is to try to define it as an 
intersection index with the submanifold =(M) corresponding to some class 
= Cc V of orbits of the action of G on V. The closure =(M) may not be 
smooth and to see if the intersection index with =(M) is well defined we 
need to study the singularities of =(M) of codimension 1. Moreover, if we 
need to define an integral cohomology class rather then the class mod 2 we 
should define a coorientation of =(M). To do this, V.Vassiliev introduced 
in [V] two chain complexes, an integral complex w of coorientable classes 
and a complex v of Z2-modules of all classes. Generators of these universal 
complexes are formed by the orbit classes of some cellular classification and 
the differential is given by adjacencies of classes of neighbor codimensions 
(see [V] for the details). 


4.1. Theorem. The row 


6 6 6 
0 ee oe ph p20 21, 


of the characteristic spectral sequence coincides for the coefficient groups 
A=Z and A= Zz with the Vassiliev universal compler w of coorientable 
classes and the complez v of all classes respectively. 


4.2. Corollary. Terms E> © of the characteristic spectral sequence coin- 
cide with the p-th cohomology group of the Vassiliev complez w (for A = Z) 
and that of the complex v (for A = Zz). 
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The proof follows from Theorem 3.10, the description of the groups 
H°(BGz, Oz) which are isomorphic to A if the sheaf Oz is trivial and 
H°(BGz, Oz) = 0 if not, and from the definition of the universal complexes 
given in [V]. 


The dimensional arguments show that the higher differentials 6,, r > 2, 
are trivial on terms E?-° and the following homomorphisms are well defined 


H?(w) ~ ES —. Ep° —, H?(BG) — H?(M) 


(for the integral coefficients; the same holds for the complex yv and Zo- 
coefficients). Thus, the cohomology classes of the Vassiliev complex define 
characteristic classes. The homomorphism obtained H?(w) — H?(BG) 
may have a nontrivial kernel which corresponds to the nontrivial differ- 
entials 6,: E2-""—1 — EP), r > 2. It follows that the cohomology class 
defined by some nontrivial class of the Vassiliev complex may be always 
trivial for any (G,V)-bundle and its section on M, and this fact can be 
established only by considering adjacencies of classes the codimension of 
which differ by more then 1. Examples of such phenomena are given in 
[K2] and in Sect.5 below. 


5. Characteristic spectral sequence 
of right equivalence of functions 


It would be interesting to apply the theory described in the previous 
sections to the concrete examples of Lie group actions. Nevertheless, the 
main point of our interest belongs to the infinite dimensional cases of classi- 
fications which appear in the singularity theory. In this section we consider 
the case when G is the group of diffeomorphism germs of (R”,0) and V is 
the space of smooth function germs (R”,0) — (R, 0). 

More precisely, denote G(k,n) the group of k-jets of diffeomorphisms 
of the space (R”,0) and V(k,n) the space of k-jets of functions (R",0) — 
(R, 0). Denote EY" (k, n) the characteristic spectral sequence for the natural 
action of G(k,n) on V(k,n). 

Let k’ > k be two integer. Then if k’-jets of two functions are G(k’,n)- 
equivalent then their k-jets are G(k, n)-equivalent as well. Hence, the clas- 
sification on the space of k’-jets of functions may be obtained as a subpar- 
tition of some classification on the space of k-jets of functions and we get 
a homomorphism of spectral sequences 


TE : EP (k,n) —> EP9(k!,n) 


5.1. Proposition. For any integer d there is a number k(d) such that 
the homomorphisms 7 above are isomorphisms for any fired p, q, r, n, 
and k'>k > k(p+q). 
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The proof is similar to that of the corresponding proposition in [K2] 
for the case of the stable right equivalence. 


5.2. Definition. The limit spectral sequence 
EP-4(n) = lim EP4(k,n) 
° k—- 00 
is called the characteristic spectral sequence of right equivalence of functions 
of n variables. 


The group G(k,n) is contractible to the group G(1,n) = GL(n,R) of 
linear diffeomorphisms. Hence we have 
5.3. Theorem. The characteristic spectral sequence of right equiva- 


lence of functions converges (for the case of Zo-coefficients) to the ring of 
polynomials of Stiefel- Whitney classes w1,...,;Wn of degrees 1,...,n. 


The following theorem give an explicit description of the limit term 
EP24(n). 


5.4. Theorem. For k = 1 we have 


0 , p#O andpFn 
EP4(1,n) ~ EB@(1,n) ~ ¢ H4(BO(n—1)), p=0 
HUBO(n)) , p=n 


The natural homomorphism 
7: EP4(1,n) —> EP(n) 
for r = co induces an isomorphism of the limit terms E24. 


Proof. The action of the group G(1,n) on V(1,n) is the standard 
action of GL(n,R) on R”. Therefore, the first part of the theorem is a 
reformulation of Theorem 2.2 as it is explained in Example 3.8. 


5.5. Lemma. For r = 1 the homomorphism 
ee ee a) 
is an isomorphism for p <n and a monomorphism for p = n. 


Proof. For p = 0 the groups BP (ke, n) correspond to the orbit of k-jets 
of nondegenerate functions. A nondegenerate function is right equivalent 
(up to a constant) to a linear function and the stationary group of its k-jet 
contains the subgroup O(n —1) of orthogonal transformations of the hyper- 
plane given by zeros of the function (with respect to some fixed Euclidean 
structure). The inclusion of this subgroup is a homotopy equivalence, as 
one can prove, and we have 


E®*(1,n) ~ Ep" (k,n) = Ey"(n) ~ H*(BO(m — 1)). 
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Now, for 1 < p < n there is no orbits of codimension p of the action of 
G(k,n) on V(k,n) and we have 


BE (k,n) = 2 (ay Pp <n 


For p = n the groups Ej'*(k,n) correspond to the orbits of k-jets of Morse 
singularities. A germ of function with a Morse singularity is right equivalent 
(up to a constant) to a nondegenerate quadratic form. Consider the orbit 
for which this quadratic form is positive definite and, hence, defines a Eu- 
clidean structure. For any k > 2 the stationary group of its k-jet contains 
the subgroup O(n) of linear orthogonal transformations and the inclusion 
of this subgroup is a homotopy equivalence. Hence, in accordance with 
Theorem 3.10 one of the direct summands of the group Ey 1 is isomorphic 
to the group H*(BO(n)) which shows that 


TP: EY*(1,n) = H*(BO(n)) + Ey" (k,n) ke. 


is monomorphic. This completes the proof of Lemma 5.5. 

Using Lemma 5.5 we prove the second part of 5.4. Denote I C E{”*(n) 
the image of 7: E}"(1,n) — Ej’*(n). According to Lemma 5.5 and the first 
part of Theorem 5.4 all the differentials 6, vanish on J. The dimensional 
arguments show that J does not intersect the image of any differential 6,. 
Hence, J maps monomorphically to the limit term E%;*(n). But, according 
to Theorem 5.3 the ranks of the groups E3;*(1,n) and those of E*:*(n) 
coincide. Therefore the monomorphism 1: E2:*(1,n) > E*:*(n) is, in fact, 
an isomorphism. 

This proof looks rather artificial. In 5.6 we give its geometrical meaning. 

Consider an n-dimensional vector bundle 7:E — M and a generic 
function f: # — R in the total space defined in some neighborhood of the 
zero section. With every point z € M we associate a function germ of 
the restriction of f to the fiber 7~1(z) & R”. The classification of critical 
function points give a classification of points of the manifold M. This give 
a spectral sequence E,’*(M) converging to the cohomology group of M and 
a homomorphism 


KY. BY" (n) > Ep*(M) 
such that the limit homomorphism of the cohomology groups 
k*: H*(BO(n)) — H*(M) 


evaluates the characteristic classes of the vector bundle z. 
Theorem 5.4 have the following unexpected corollary which can be 
considered as its geometrical meaning. 


5.6. Proposition. Any characteristic class defined in terms of non- 
Morse singularities is trivial. 
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Here we give the direct simple proof. The proposition follows from 
the observation that for every vector bundle 7: E — M there is a function 
fo: E — R such that all singularities of its restriction to the fibers mee) 
z € M are of Morse type. To build such a function one should take any 
generic section h of the conjugate bundle E* — M considered as a function 
on E with linear restrictions to the fibers of 7 and any Riemannian structure 
g on E considered as a function with quadratic restrictions to the fibers of 
mw. Then he function 


fo=h+g9 


have the properties as required. 


5.7. Example. An n-dimensional foliation on a smooth manifold M is 
an integrable n-dimensional distribution v that is an integrable subbundle 
of the tangent bundle. n-dimensional integral submanifolds of this distri- 
bution are called fibers. 

Let f:M — R be a generic function. Then its restriction to a generic 
fiber is a Morse function, but non-Morse singularities of the restriction of 
f to the fibers can appear irremovably by small perturbations of the func- 
tion. In [AGLV] it is suggested to build characteristic classes of foliations as 
classes dual to the cycles formed by points of some singularity type of the 
restriction of f to the fibers. Theorem 5.3 stays that the Stiefel-Whitney 
classes of the bundle v only can appear in such a way. Moreover, in ac- 
cordance with Proposition 5.6 any characteristic class defined in terms of 
singularities which are more complicated than Morse ones is trivial. 

Let E be the total space of the bundle v. Then there is a smooth map- 
ping Exp: E — M defined in a neighborhood of the zero section which is 
identical on the zero section and which maps diffeomorphically the fibers of 
the bundle v to the fibers of the foliation. Then the function Exp*f:  - R 
produces the same classification of points of M in accordance with singu- 
larity type as the function f itself. This explain why the integrability of 
the distribution v does play any role. 


5.8. Although the limit term of the characteristic spectral sequence 
has a simple description in Theorem 5.4 the spectral sequence itself has a 
rather complicated behavior and seems to have no good description in large 
dimensions. For example, the cohomology groups of the Vassiliev complex 
which correspond to he row Ee of our spectral sequence are calculated 


in [V,AGLV]. For | < 6 the groups Ent}(n) for A = Z and A = Za, 
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respectively, are given in the following table 


Generators of these groups are described in the following theorem. 


5.9. Theorem ([AGLV]). For any n-dimensional foliation on M and a 
generic function on it the following chains equipped with proper signs form 
integral cycles: "At U TA PAR Ue. G) "45, "Pi U"P?2. In addition, there 
are the following cycles mod2: AJ U...U""4,, AF u...u™-M7. 


The left superscript in this theorem denotes the number of positive 
squares of the second differential of a function germ, see [V], [AGLV] for 
the notations. As a corollary of Proposition 5.6 we have 


5.10. Theorem. All the cycles listed in Theorem 5.9 are homological 
to zero. 


To give an explicit description of this phenomenon one should consider 
adjacencies of singularities the codimension of which differs by more than 
i 

As an example we prove the relation Af U As ~ O for the case 
n = 1. The same arguments explain the relation As, aU Aga 
In fact, our proof contains the calculation of the (nontrivial!) differential 
Ra oe E3*°(1). One could give similar description for the other 
cycles from Theorem 5.9. 

Consider a smooth linear bundle 7: E > M and a function f:E SR 
given in a neighborhood of the zero section. Without loss of generality 
we can assume that M is 4-dimensional and compact and f is generic. In 
particular, the set of singularities ae of the restriction of f to the fibers of 
m consists of finite number of points. We show that this number is always 
even. 

The closure A3(M) C M of the set of points of singularity type A} is 
smooth and two-dimensional. The closure of A, divides it into two domains 
Az (M ) and A;(M). For any closed curve y C At+(M) we define its index 
At (7) € Zz which is equal to 0 or 1 depending on weather orientable or not 
the restriction of the bundle m to y. This index does not change when we 
exchange the curve y by the homological one and defines the cohomology 
class [AJ] € H'(A}(M)) which is nothing but the first Stiefel-Whitney 
class of the restriction of the bundle 7 to A3(M) C M (hereinafter by the 
cohomology we mean the cohomology mod 2). 
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Near the points of the type A, the function with singularity An at the 
origin has nearby one more critical point. The direction to this eacionel 
point defines a particular orientation of the restriction of 7 to AJ(M) in 
a neighborhood of the curve A4(M). This allows to extend the index Af 
to the set of curves in At(M) with ends in A4(M): it is equal to 0 or 
1 depending on weather this particular orientation passes to itself or not 
when going along the curve from one its and to another. Define now the 
index [At] € H1(A3(M) \ {AS (M)}) which on a closed curve y C A3(M) \ 
{AF(M (M)} is equal to the At-index of the intersection of - with the closure 
Ad(M). 

Let C Cc At(M) be the union of small circles around all A¥-points. 
Then C is a boundary of a 2-chain in A3(M) \ {45 (M )}, and hence, the 
sum of At -indices of small circles around all Ae -points is equal to zero 
(mod 2). It remains to observe that the At index of a small circle going 
around any point of the type ae is equal to 1. The proof of this is seen 
from Fig.1. 


\ 
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Figure 1. Functions with singularity Gs 


at the origin near a function with singularity A= 
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Value of generalized hypergeometric 


function at unity 


A. Kazarnovski-Krol 


Abstract 


Value of generalized hypergeometric function at a special point is calculated. 
More precisely, the value of certain multiple integral over vanishing cycle (all 
arguments collapse to unity) is calculated. The answer is expressed in terms of 
[-functions. The constant is relevant to the part of p in the Gindikin-Karpelevich 
formula for the c-function of Harish-Chandra. Calculation is an adaptation of 
classical calculations of Gelfand and Naimark (1950) to the Heckman-Opdam 
hypergeometric functions in the case of root system of type An-1. 


0. Introduction 


In this paper the value of certain multiple integral over a vanishing 
cycle (all arguments collapse to unity) is calculated. This is a normalization 
constant. It is related to the part of p in the Gindikin-Karpelevich formula 
for the c-function of Harish-Chandra. The cycle is a distinguished one in 
the theory of zonal spherical functions. It appeared for the first time in (1] 
and its role is clarified in [21]. 

Technically our calculations are an adaptation of formulas of [1] (el- 
liptic coordinates) for the case of generic k, i.e., for the Heckman-Opdam 
hypergeometric functions of type An-1. The integral may be regarded as a 
variation on a theme of the Selberg integral [19], and recently integrals of 
this type have drawn much attention because of applications to conformal 
field theory. 

The paper has continuation in {20,21}. 


1. Theorem 


Letezele= 1c ti, @ = tees, 9 — 1,...,.% — 1 be the set of 

real variables. It is convenient to organize these variables in the form of a 
pattern cf. fig.1. 
Remark 1.1 This type of variables reflects the flag structure (ith row cor- 
responds to i-dimensional plane in a flag), cf.[1,2]. One should notice that 
this type of variables is used in [10] for the isomorphism of Heckman-Opdam 
hypergeometric system with a particular case of trigonometric version of 
Knizhnik-Zamolodchikov equation. 
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2 22 406 oe Zn 


tin-1 ton—1 1a tr-1,n-1 


ti2 t2.2 


tia 


Figure 1. Variables organized in a pattern 


Definition 1.2 Consider the following multivalued form WA? 


mT 
wa =|] 2 [] (2, - zi)-* 
t=1 


11 >%2 
kel k-1 
x [[ (2 - tin-1) [] Gin-1 — 2) 
i<l i>l 
n—-2 n—2 
k-1 k-1 
x TT [] Gag - tag) TT UT (saa - tas) 
J=1 ty Diag j=l i1>%2 
n—1 
2-2k 
x] [] Gas - tay) 
j=2 iy >ig 
n~-1 j 
aiz 
x II Il ti dty1dti9dtoo a68 att 
f= gai 


Remark 1.3 For the applications one should let Qig = An—j41 —An-j7 —k 
and a; = 4; + koe), but we formulate the theorem in this more general 
form (with a;; and a; unspecified). 

Definition 1.4 Set 


Bary s2n)= f wa, 
A 


where wa is defined above and , assuming that 21,22,...,2n are real and 
satisfy 0 < 21 < 22 <...< zn, define cycle A by the following inequalities: 
bigti Stig S tiga jg, and 2 < tani < 241 . 

We assume that phases of the factors of the form Wa are equal to zero 
provided k , a;;, and a; are real. 


Theorem 1.5 The limit of (z,... ,2n) as all z; approach 1, while z, < 
22 <... < 2y, ishegual to: 


(kK) (k)?... E(k)” 


BLD Bim B15... 429) = Pera Te 


all z;—1 
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Proof. Following the classical work of I.M. Gelfand and M.A. Naimark cf. 
[1] , let 

fa 

UT a ji — ty) 

Wh= 


(Gee) 


aT ft 


inh = 


j 
1 (eee — gt | Notesthat, © 7; = 1, and 


Sil 
(ti,j-1 — tk,j—1) 
Dice nis) isch ; 
DG ei; 15=1) I] (tis — tp;) 
1<i<p<j 
[see [1] for the details]. Let also 
n—-1 
I] (a ,n-1 — 2) 
=a : 
 — See C= lena sie 
i I] (2a — 24) oi 
2 8 
mT 
One has )>Tin = 1 and 
ail 
Gate 1) 
DGG ne, 1S 
WOE 19 arias tne I] ea = Zp) 
l<i<p<n 


In the variables 7; i=1,...,j-1, j=1L.-. ,nintegral for ®(21,... ,2n) 
is written as: 


Gis 27) — \e / 
i=1 


nr 
x [[ (us: p00 easing ial eG os T)-1,3))* “dti2dr13d723 one dT1n ee din=in a 


j=l 


nr 


-1 j 
iGae 
1 


j=l i= 


Integration is taken over 


jal 
Tj > 0, Weg, <auille 
ai 
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As all z;, 1 =1,... , approach 1, all ¢;; also approach 1, so 


. k-1 
(1, ric «l) =| iL s< [[(a57; O6 Peper ele) = Ooo — Tan 
j=l 
x dt 2d713dT23 900 atin oa6 Cie 


So using Dirichlet’s formula one gets 


Zz n 

Scien D(k)D( Se... (k) 
[(k)D (2k)... T (nk) 

Remark 1.6 The same calculation as in theorem 1.5. shows that 
®(z), Z2,.-- , Zn) is equal to the same constant as in theorem provided that 
all a;; = 0 and a; = 0. Then this constant is interpreted as the volume 
of a maximal compact subgroup (under certain normalizations)cf. [1,13,14] 
in the case of generic k, i.e. when there is no group and no subgroup at 
all. The fact that this constant does not depend on z; also implies the 
monodromy properties of cycle A, cf. [21]. 
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Harish-Chandra decomposition for 
zonal spherical function of type 4, 


A. Kazarnovski- Krol 


0. INTRODUCTION 


Heckman-Opdam system of differential equations is holonomic , with 
regular singularities and has locally |W|-dimensional space of solutions (Cot: 
corollary 3.9 of [12]), where |W| is the cardinality of the Weyl group W. The 
system is a generalization of radial parts of Laplace-Casimir operators on 
symmetric Riemannian spaces of nonpositive curvature and is isomorphic 
to Calogero-Sutherland model in the integrable systems. 


Harish-Chandra asymptotic solution is a unique solution of the system 
with the prescribed asymptotic behavior: 


eee lee) 


(0 < |x| < |z2| <... < |2ntil). Here w € W are elements of the Weyl 
group. These solutions provide a basis in the space of all the solutions in 
the chamber 0 < |z;| < ... < |Zn4i|. Among all the solutions there is a 
distinguished one up to the constant multiplier, which admits continuation 


to analytic function at z1 = 22 = 23 =... = 2n41 #0. This solution is re- 
ferred to as zonal spherical function. Zonal spherical function is normalized 
s.t. it is equal to 1 at z1 = 22 =... = 2n41 = ie 


Representation of the zonal spherical function as linear combination 
of elements of the basis (Harish-Chandra asymptotic solutions) is called 
Harish-Chandra decomposition. 


In ref. [11] we provided an integral representation for the solutions of 
Heckman-Opdam system of differential equations in the case of A,. We also 
described contours for integration A,,, integrals over them provide Harish- 
Chandra asymptotic solution F,,(z). In ref. [34] we studied the cycle A 
for integration for zonal spherical function . This paper is devoted to ho- 
mological treatment of Harish-Chandra decomposition for zonal spherical 
functions of type An. Namely, we explicitly decompose the distinguished 
cycle A cf. [10 ,16 } into linear combination of cycles A,, described in [11] 
and check that after normalization this turns out to be the Harish-Chandra 
decomposition for zonal spherical function of type An (theorem 2.2 and 3.1 
below). The point of view that linear relations between the solutions re- 
flect the linear relations in homology group is due to B. Riemann. He also 


OThis paper is a revised version of the preprint of the author “Decomposition of a 


cycle” 
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emphasized the importance of the monodromy. In this case the correspond- 
ing homology theory is described in [2 , 37]. Harish-Chandra asymptotic 
solutions correspond to conformal blocks in conformal field theory (WA, 
algebras) and provide a basis in the space of conformal blocks, zonal spheri- 
cal function is a particular conformal block, in the case of Ag see figs. 3a,3b, 
3c,3d,3e,3f and 2 below. 


0.1 Notations. 

@1,@2,... ,@p, - simple roots of root system of type A, 

Aj, A2,... ,An+41 - weights of fundamental (vector) representation, 
Ag = Ay — ay, A3 = Ay — Q@1 — Q,... , Ay + Ag+...+Ansi = (0), 

R4 - set of positive roots 

6=} ee rp, @ -halfsum of positive roots 

k- complex parameter (‘halfmultiplicity’ of a root) 


p=5 oe 


a€Ry 
c(A, k)- c-function of Harish-Chandra 


1. MULTIVALUED FORM 


Consider the following set of variables: 

21, — i n+, taj, Caer nals gH... , 7. 

Variables z; have meaning of arguments, while variables t,; are variables 
of integration. 

It is convenient to organize variables z;, tij in the form of a pattern, 
cf. fig 1. 


Z1 22 ono on Zn+1 


ie ton _ oo 


FIGURE 1. Variables organized in a pattern 


The idea of such an organization is borrowed from Gelfand-Zetlin pat- 
terns [1]. 


Definition 1.1. Consider the following multivalued form w(z,t): 
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n+1 es 
w(z,t) = Il Zp hd Il (Ge = al 

al) ty >t2 

n+l n, 

k— 

x Il Il (zi — ae : 

=i Gil 

n—1g+l 7 


ike aa 


geal inl 


nr 
<A Cap atnae 


j=211>%2 


Al 
An—j42—An—j4i-k 
x Il Il oa? je dt,,dtj9dt2 588 ding 


Remark 1.2. k is a complex parameter - ‘halfmultiplicity’ of a restricted 
root, cf. Heckman, Opdam [12]. 

In [11] we proved that integrals over the form w(z,t) over appropriate 
cycles provide all the solutions to Heckman-Opdam system of differental 
equtions and described cycles A for Harish-Chandra asymptotic solutions 
( definition 4.3 and theorem 6.3 of [11]). 


Definition 1.3. A complex number z can be represented as z = re, 
where r,a@ are real numbers, r > 0. r is called absolute value of z, while a 
is called the phase of z. When we say that the phase of a complex number 
z is equal to 0, we mean that a = 0, or the number itself is real and 


nonnegative. 


2. THE DISTINGUISHED CYCLE A 


Assume that z1,22,.-- )Zn41 are real and 


0< 2 <2%<...< 2n41- 


Definition 2.1. Define cycle A = A(z) by the following inequalities: 
tage S ti S tiga gi and 
2 lin Al 
Define form wa (z, ¢) as: 
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n+1 res eer 
LYNE) = I] Bete II Gaon 
i=1 41 >22 
_ k-1 

x ]] = tin) T] Gin — 21) 

i<l i>l 

+ k-1 

j=l t1>%2 ig>ty 

Tm 

—2k 
JH2 t1>%2 


n Jj 
—j+2—An—j4i—k 
x ils 552 aoe dt, dtj2dtao...dtnn 
j=li=1 


It is assumed that phases of factors in the formula for wa (z, t) are equal 


to zero if k and Aj, A2,-.-. ,An41 are real. For the homological meaning of 
the cycle A see fig. 2 below and theorem 5.7 of [34]. 


Ay Ay 
Ay 


Ay + Ag 


Ay +Ag+A3=0 


A-6 Soom ae A-~6 


FIGURE 2. Zonal spherical function 


In [11] cycles A,,(z) and forms ur, (z,t) were described,in the case of 
Ag see figs. 3a,3b,3c,3d,3e,3f below. In the case A, the figures are similar. 


The following theorem explains the relation between A(z) and A,,(z). 
Theorem 2.2. (Harish-Chandra decomposition ) 
‘) Wa(et)= >> d(w,r,k) | Wway(z), 
A(z) wESn41 4u(z) 


where 
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e2Fi(A,6) om I(w)(k—1) 


(21) IT sin(—7(wA, av)) 
ae hy 


b(w, A, k) = 


Ay Ay Ay 


= A-6 
A—6+Ai + Az 
G 2 - 
I ee 
FIGURE 3A. 
Ay Ay Ay 
a : A-6 
N= 6+ Ao A—~ 6+ Ag+ Ai 


ee) 
Zl 3 


FIGURE 3B. 
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NS Oo Age 


(; 2 ) 
Bh 


SOAS As 


& 2 4 
23 1 


FIGURE 3D. 
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7 rA—6 
A—6+A, + Az 
(; 2, ) 
rs 2 
FIGURE 3E. 
Ay Ay Ay 
A—6 


AN—S+A, + Ae 


ees 
2a 


FIGURE 3F. 


The theorem is an application of the two following lemmas, also section 
2 of [11] is useful. 
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Lemma 2.3. (Elementary decomposition) Let 2), 2z2,...,2n be real and 
0 < Zz < 22 <...< 2). Consider the following integral: 


[eG St) (ot) ae dt 
and consider contours 1 (t), y2(t), y(¢), ¢ € [0,1] as follows. 
y(t) = b2e— 5 =P (1 = t)z; 


71(t) is a loop which starts and ends at z;-1 and goes counterclockwise s.t. 
the following inequalities are fullfilled for all t € (0, 1]: 


Zi? <i) | See 


Y2(t) is a loop which starts and ends at z; and goes counterclockwise ,s.t. 
the following inequalities are fullfilled for all t € [0,1]: 


Ze yelt)| Se, 


as indicated on fig. 4, phases of the factors should be appropriately chosen. 
The following is the specific choice of the phases : if all ap,a1,... ,Qn are 
real, then the phase of the integrand along 7;(t), y(t), y(t) is chosen to be 
zero for small values of t. 


FIGURE 4. Elementary decomposition. 


Then we have the following relation between -y, Yo and +: 


e—7(ao+a1+...+a;-1) e—7t(@0+a1+...+a:) 


= Us 
: a (27) sin m(ap + a1 +... + a;) ue (22) sin t(a9 +a, +...+;) 


Lemma 2.4. (Elimination of ‘wrong’ diagrams). Integrals of the form 
w(z,t) (ww(z,t)), such that contours for integration Of taj, tijsa, tiga,j4) 
are shown on fig. 5, provided k is not an integer, are equal to zero. We 
suppose that tj goes from t j41 to Upmes eei| ; ti j41 goes from bid j+2 to 
tin j4+2> and ti41,j41 goes from ti41 742 to bisa j+2 Oe fig. 5a. 
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The same holds true for t:-1n—-1,ti-1,n, tin, and 2 correspondingly, ce 


fig. 5b. 


titt,jt 


ti41,j+2 


se (b) 


Ficure 5. (a), (b), Cycles of this type are homological to zero. 


By the ’wrong’ diagrams we mean diagrams ,where the two arrows have 
the same target, see figs. 5 and 6 of [34] . 


Remark 2.5. Lemma 2.4 is equivalent to quantum Serre’s relations in the 
form given in [3], see also [2]. 


3. NORMALIZATION 
Let 


T'((—wa, aY)) sin(r(—wa, a”) 
w(2) = ( I] T((—wd, a’) +k) 


aeéRy, 
seen 2A) AiR INCA) T (RYE (24) =a Ww(2,t) 
Aw(z) 


Then 


F(z) = guerra...) 
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cf. [11] theorem 6.1. 
Also, let 


5) — E(RE 2k)... Pn + 1k) 


Fa ( ) T(k) EH +2) wa(z, t) 


A(z) 

When Fa(1,1,.:. 1) = 1) ef. [10] thearemmies: 

After this normalization theorem 1 reads as usual Harish-Chandra de- 
composition cf. [12,15]. 
Theorem 3.1. In the above normalization we have: 


Fa(z)= > c(wdA,k) F(z), 


wESn+1 
where c(wA,k) is a c-function of Harish-Chandra: 


P'((p,a”)+k) 
Taek, Tay) 

n(n: 
Tacr, Twi) 


c(wA, k) = 


Le. Fa(z) is identified with zonal spherical function. 


Corollary 3.2. Suppose z(t), zo(t),... , 2n4i(t), t € [0, 1] are closed loops 
on a complex plane, i.e. z(0) = z(1),z2(0) = 29(1),... Keeani (0) = 
2n41(1), such that z;(t) 4 z;(t) fori # j. Let also Re(z,(t)) > 0 for each 
a=1,...,n+1. Then the homological class of the cycle A is preserved 
under the monodromy along paths Zt): 


Remark 3.3. In this approach multiplicative structure of c-function of 
Harish-Chandra gets a very simple explanation. Namely: 


Winco (ES) 
Thsicicn Tee) deere tem eee 


Tl _ T((—wA,ei—e; )+k) s I T((—wa,ei—en41)+k) 
1<i<j<n a(S sa 1si<n+1  T((—wd,e:—e€n41)) 
Here {e; =o ss =< 7 Si 1} are positive roots of root system of 
type An. Multiplicative properties of c-function of Harish-Chandra were 
observed by Bhanu-Murti in the case of SL(n,R) and in general case by 
Gindikin and Karpelevich [17]. c-function of Harish-Chandra is equal to 
the product of elements of 6j-symbols , see fig. 6. Multilpicative structure 
of c-function of Harish-Chandra amounts to simple combinatorics related 
to positive roots , in this case: 


CO) = 


{e:—ej|1 Si <j <ntl} = {e,-e,|1 <i <j < nr} (J f{ei-ensill <i < n}. 


This combinatorics is both very instructive and restrictive. 


Harish-Chandra decomposition 357 


Ay Ay 
Ay 


Ai +A A 


—_—_—___—__> 
Ay + Ag+ As Ay + Ag | 
-6 = = et > — A-6 


—— 
A-—6+Ai+ Ag 


FiGurE 6. c-function of Harish-Chandra as a product of elements of 6j-symbols . 


Remark 3.4. We have also checked the monodromy properties of the cycle 
A using quantum group argument, see [34]. 


Remark 3.5. Harish-Chandra decomposition for zonal spherical function 
might be considered as an analogue of Bernstein-Gelfand-Gelfand resolu- 
tion. 


Concluding remark. We would like to point out once more that the 
distinguished cycle A appeared in the classical calculation of Gelfand and 
Naimark [16] of zonal spherical function for SL(n,C), originates in the so- 
called elliptic coordinates and provides a materialization of the flag mani- 
fold. 


Acknowledgments. I am grateful to I. Gelfand for stimulating discus- 
sions concerning the theory of spherical functions and the theory of hyper- 
geometric functions, to S. Lukyanov for stimulating discussions concerning 
conformal field theory, to V. Brazhnikov for helpful discussions. 
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Positive paths in the linear symplectic group 


Francois Lalonde* Dusa McDuff 


1 Introduction 


A positive path in the linear symplectic group Sp(2n) is a smooth path 
which is everywhere tangent to the positive cone. These paths are generated 
by negative definite (time-dependent) quadratic Hamiltonian functions on 
Euclidean space. A special case are autonomous positive paths, which are 
generated by time-independent Hamiltonians, and which all lie in the set U 
of diagonalizable matrices with eigenvalues on the unit circle. However, as 
was shown by Krein, the eigenvalues of a general positive path can move off 
the unit circle. In this paper, we extend Krein’s theory: we investigate the 
general behavior of positive paths which do not encounter the eigenvalue 1, 
showing, for example, that any such path can be extended to have endpoint 
with all eigenvalues on the circle. We also show that in the case 2n = 
4 there is a close relation between the index of a positive path and the 
regions of the symplectic group that such a path can cross. Our motivation 
for studying these paths came from a geometric squeezing problem [16] in 
symplectic topology. However, they are also of interest in relation to the 
stability of periodic Hamiltonian systems [9] and in the theory of geodesics 
in Riemannian geometry [4]. 


Main results 


We consider R2” equipped with the standard (linear) symplectic form 
w(X,Y)= Vie 


where J is multiplication by 7 in R2" = C”. Thus if e1,...,€2n form the 
standard basis Jeg;-1 = e2i. The Lie algebra sp(2n) of G = Sp(2n, R) is 
the set of all matrices which satisfy the equation 


AT J+ JA=0. 


_————— 
*Partially supported by NSERC grant OGP 0092913 and FCAR grant ER-1199. 
tPartially supported by NSF grant DMS 9401443. Both authors supported by NSERC 

grant CPG 0163730. 
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Hence A € sp(2n) if and only if JA is symmetric. The symplectic gradient 
(or Hamiltonian vector field) Xy of a function H : R?” — R is defined by 
the identity 


WN, 4) YX de ton alla et 


Given a symmetric matrix P, consider the associated function 
1 + 
Ox) = —5 1, 


on R?". Then the calculation 
dO Yi -Y Pry JUPz) 
shows that the symplectic gradient of Q is the vector field 
Die) = JP) eaten = ee 


which integrates to the linear flow tz »% e/Ptz. We call these paths 
Vee *}+e[0,1] autonomous, since they are generated by autonomous (i.e. 
time-independent) Hamiltonians. 

We are interested in paths A; € G which are positive in the sense 
that their tangent vector at time t has the form JP,A, for some symmetric, 
positive definite matrix P,. Thus they are generated by a family of time- 
dependent quadratic Hamiltonians! of the form Q, = —sa7 Pyz. These 
paths are everywhere tangent to the positive cone field P on G given by 


BiaeG ear Uwce (iP Ave — P? PS 0) ery gice— 16) 


In particular, an autonomous path e7”* is positive if P is. 

Note that an autonomous path eJ?* always belongs to the set U of 
elements in G which are diagonalizable (over C) and have spectrum on the 
unit circle. The following result shows that this is not true for all positive 
paths. 


Proposition 1.1 Any two elements in G may be joined by a positive path. 


This is a special case of a general result in control theory known as 
Lobry’s theorem: see Lobry [17], Sussmann [19] and Grasse-Sussmann [11]. 
We give a proof here in §4.2 as a byproduct of our other results. 

The structure of positive paths Ate{o,1) in G was studied by Krein in a 
series of papers during the first half of the 50’s. Later Gelfand—Lidskii [9] 


It is somewhat unfortunate that our sign conventions imply that Q; is actually 
negative definite. 
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took up the study of the topological properties of general paths in con- 
nection with the stability theory of periodic linear flows. Almost simul- 
taneously, and surely independently, Bott [3, 4, 5] studied positive flows 
in the complexified linear group in connection with the geometry of closed 
geodesics. As a result of these works, the structure of positive paths is well- 
understood provided thatthe spectrum of A; remains on the unit circle. 
However, we need to investigate their behavior outside this circle. Sur- 
prisingly, a thorough study of the behavior of these paths outside the unit 
circle does not seem to exist. The aim of this article is to undertake such 
a study, at least in the case of generic positive paths, that is to say those 
paths which meet only the codimension 0 and codimension 1 strata of the 
matrix group. For the convenience of the reader we will develop the theory 
from scratch, even though some of our first results are well-known. 

Our main result is motivated by the geometric application in [16]. Be- 
fore stating this result, we introduce some notation. As above, U denotes 
the set of elements in G which are diagonalizable and have spectrum on the 
unit circle (unless indicated to the contrary, “diagonalizable” is considered 
over C). Further, 


S; = {AEG: det(A— 1) = 0} 


is the set of elements with at least one eigenvalue equal to 1. A path 
y = Atejo,y in G which starts at the identity Ap = 1 and is such that 
A, € G—S, for all t > 0 will be called short. Equivalently, it has Conley- 
Zehnder index equal to 0.2 Moreover, an element A €G is called generic 
if all its eigenvalues (real or complex) have multiplicity 1. In the next 
theorem, all paths begin at 1. 


Theorem 1.2 (i) An element of G—S, is the endpoint of a short positive 
path if and only if it has an even number of real eigenvalues with 
ee. 


(ii) Any short positive path may be extended to a short positive path with 
endpoint in U. 


(iii) The space of short positive paths with endpoint in U is path-connected. 


We can interpret these results in terms of a “conical” subRiemannian 
geometry. SubRiemannian geometry is usually a study of paths in a mani- 
fold M which are everywhere tangent to some distribution € in the tangent 
bundle TM. For example, there is recent interest in the case when € is a 
contact structure: see [12]. One could generalize this to the consideration 


ee 

2The Conley-Zehnder index (or Maslov index) of a path measures how many times 
it goes through the eigenvalue 1 and characterizes its homotopy class: see Gelfand- 
Lidskii [9], Ekeland [7, 8], Robbin-Salamon /citeROSA1. 
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of paths which are everywhere tangent to some fixed convex conical neigh- 
borhood N,(€) of €. The geometry of positive paths more or less fits into 
this context: one just has to replace the distribution € by a distribution of 
rays. Thus we can consider the positive cone to be a neighborhood of the 
ray generated by the right invariant vector field JA € T,4G corresponding 
to the element J in the Lie algebra. Proposition 1.1 shows that G re- 
mains path-connected in this geometry. Next, one might try to understand 
what happens to the fundamental group. More precisely, let us define the 
positive fundamental group 71,p0;G to be the semigroup generated by 
positive loops based at Jl, where two such loops are considered equivalent 
if they may be joined by a smooth family of positive loops. It would be 
interesting to calculate this semigroup. In particular it is not at present 
clear whether or not the obvious map 


71 posG = fl (G) 


is injective in general. (When n = 1 this is an easy consequence of Propo- 
sition 2.4 and Lemma 3.1.) 

The proof of Theorem 1.2 is based on studying the relation between 
the positive cone and the fibers of the projection 7: G > Conj, where Conj 
is the space of conjugacy classes of elements in G with the quotient (non- 
Hausdorff) topology. The space Conj has a natural stratification coming 
from the stratification of G. Recall that a symplectic matrix A is simi- 
lar to its transpose inverse (A7)—1 (in fact, (AT)-! = —JAJ). Thus its 
eigenvalues occur either in pairs \,\ € S! and \,1 /X € R—0O, or in com- 
plex quadruplets 4, ,1/A,1/\. In particular, the eigenvalues +1 always 
occur with even multiplicity. Roughly speaking, the open strata of G con- 
sist of generic elements (ie diagonalizable elements with all eigenvalues of 
multiplicity 1), and the codimension 1 strata consist of non-diagonalizable 
elements which have one pair of eigenvalues of multiplicity 2 lying either on 
R — {+1,0} or on S! — {+1}, or which have one pair of eigenvalues equal 
to +1: see {1] and §3 below. Note that a conjugacy class which lies in an 
open stratum is a submanifold of GL(2n,R) of codimension n since each 
pair of distinct eigenvalues (in RUS?) has one degree of freedom, and each 
quadruplet has two. 

We write Go for the set of all generic elements in G, G, for the union of 
all codimension 1 strata, and similarly Conjg,Conj, for their projections in 
Conj. We shall consider in detail only generic paths in Con}. By definition, 
these intersect all lower strata transversally. Hence the elements of these 
paths lie in Conjg for all except the finite number of times at which they 
cross Conj,. Note that the codimension of a stratum in Conj always refer 
to the codimension of its lift in G. 

The main ingredient of the proof of the theorem is to characterize the 
(generic) paths in Conj which lift to positive paths in G. We shall see in 
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Proposition 3.6 that the only significant restriction on the path comes from 
the way that eigenvalues enter, move around and leave S1. We will not 
state the general result here since it needs a certain amount of notation. 
However some of the essential features are present in the case n = 1 which is 
much easier to describe. In this case Conj is the union of the circle with the 
intervals (—co, —1] U[1, 00) with the usual topology, except that the points 
+1 are each tripled. To see this, first consider matrices with an eigenvalue 
pair \,\ € S?. It is possible to distinguish between these eigenvalues and 
hence to label the pair with an element in S 1 In higher dimensions this 
is accomplished by the splitting number described in §2. Here it suffices to 
note that the rotations through angles 9 and 2x — @ are not conjugate in 
SL(2,R), the only conjugating matrices being reflections. We also claim 
that the point 1 occurs with three flavors: plain 1 (the conjugacy class of the 
identity) which has codimension 3, and two nilpotent classes 1* € Conj,. A 
similar statement holds at —1. As explained in Lemma 3.1, a positive path 
has to project to a path in Conj which goes round the circle anticlockwise, 
but it can move along the real axes in either direction. Moreover, the 
projection of a generic positive path has to leave the circle at (—1)~ or 17 
and then, after wandering around (—oo, —1)U (1,00) for a while, enter the 
circle again at (—1)+ or 1* and continue on its way. 

The next step is to carry out a similar analysis in the case n = 2. We 
shall see that the restrictions in the behavior of positive paths in this case 
all stem from Krein’s lemma which says that simple eigenvalues on S! with 
positive splitting number must flow anti-clockwise round the circle. This 
gives rise to restrictions in the way in which four eigenvalues on the unit 
circle can move to a quadruplet outside the unit circle (the Krein—Bott 
bifurcation). As a consequence, we show in Proposition 5.4 that there are 
restrictions on the regions in Conj which can be visited by a generic positive 
path of bounded Conley-Zehnder index. 

In the last section of the paper, we will present a brief survey of the use 
of positive paths in stability theory, in Riemannian geometry and in Hofer 
geometry, and will give some new applications. 

We wish to thank Nancy Hingston for describing to us her ideas about 
perambulations in the symplectic group, which were a direct inspiration for 
our work, and Hector Sussmann for explaining to us some basic results in 
control theory. 


2 Basic facts 


2.1 General results on positive paths 


First, two elementary lemmas. A piecewise smooth path is said to be piece- 
wise positive if the tangent vectors along each smooth segment (including 
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those at the endpoints) are in the positive cone. 


Lemma 2.1 (i) The set of positive paths is open in the C}-topology. 
(ii) Any piecewise positive path may be C°-approrimated by a positive path. 


(iii) Let {A,},{B,} be two positive paths with the same initial point Ap = 
Bo. Then, given e > 0, there is 5 > 0 and a C°-small deformation of 
{A,} to a positive path {Aj} which coincides with {B,} for t € [0,6] 
and with {Az} fort >. 


Proof: The first statement follows from the openness of the positive cone, 
the second from its convexity and the third from a combination of these 
two facts. oO 


Note that statement (ii) above is a special case of a much more gen- 
eral result in control theory: see, for example, Sussmann [19] and Grasse- 
Sussmann [11]. 


Lemma 2.2 (i) Any positive path can be extended to a positive path which 
ends at a generic element. 


ii) Any positive path {A,} starting at Ap = 1 can be perturbed firing Ao 
g 
so that {A;} is generic. Moreover, if A, is generic we may fix that 
also during the perturbation. 


Proof: This follows immediately from openness. Oo 


We continue with some simple remarks about the relation between 
positive paths in G and their projections (also called positive paths) in 


Conj. 
Lemma 2.3 (i) The positive cone is invariant under conjugation. 
(ii) The conjugate {X~1A,X } of any positive path {Az} ts positive. 


(iii) There is a positive path from 1 to A inG af and only if there is a 
posttive path in Conj from 1 to 7(A). 


Proof: (i) The positive vector JPA at A is taken by conjugation to 
the positive vector X-!JPAX = X-1JPXX-14xX = J(XTPX)B at 
Be Mel AX. 

(ii) follows immediately from (i). 

(iii) Since the “only if” statement is obvious, we consider the “if” statement. 
By definition, a positive path from 7m (Il) to 7(A) lifts to a positive path from 
1 to Y-!AY for some Y € G. Now conjugate this path by Y~! and use 
(ii). oO 
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Warning Given a positive path {A;} it is not true that all paths of the 
form {X;,1A,X+} are positive. One can easily construct counterexamples 
in SL(2, R) using the methods of §3. 


Proposition 2.4 Let {A;} in G be a generic positive path joining two 
generic points Ao,A,. Then the set of positive paths in G which lift 
y = {n(At) }tefo,a] C Cong 2s path-connected. Moreover, the set of these 
paths with one fized endpoint is also path-connected. Finally, if Ao = 1, the 
set of these paths with both endpoints fixed is path-connected. 


Proof: Let {B;} be another path which lifts y = {7(At)}. By gener- 
icity we may suppose that {A;} and {B,} are disjoint embedded paths. 
Moreover, when n > 2, the projection in Conj of the positive cone at any 
point X € Go UG, of a generic path is always at least 2-dimensional (see 
Proposition 3.6). Hence we may assume that the paths {A;} and {B;} are 
never tangent to the fiber of the projection G — Conj. Our first aim is 
to define positive piecewise smooth vector fields €4,€p that are tangent to 
x~1() and which extend the tangent vector fields to the paths {Az}, {Bz}. 
To make this easier we first normalise the path B;, near the finite number 
of times t; <... < t, at which A; crosses a codimension 1 stratum. If T 
is one of these times t;, there is a matrix X7 such that Br = Xp lApX7, 
and by Lemma 2.1 (iii) we may suppose that Bp = Xp 14,Xr for t near T. 

On the other hand, since the set of tangent vectors Y € TG which 
projects to the tangent vector 7, (Ar) is an affine subbundle of the restric- 
tion TG |,-1(n(Ar)), We can find a section of this subbundle whose value 
at Ar coincides with Ar. This defines £4 over neighbourhoods of all times 
t;, which we choose small enough so that €, is positive. We then define €p 
over each such neighbourhood by the adjoint map Xp le, Xp. 

At all other times A; is generic and the map t ++ (A) is an immersion. 
Therefore, the set 


¥; = {w-(m(As)) te St S tess} 


is a submanifold. We now extend £4,€g to the whole of Y; by choosing 
sections of the intersection of the positive cone field with TY; which project 
to the tangent vector field of y. This is possible because this intersection 
is an open convex cone. Observe that 7 pro jects all integral curves of £4 
and €g to y. Indeed all integral curves of the vector field s€4 + (1 —s)&B 
(for s € [01]) are positive paths which project to y. Clearly there is a path 
of such curves joining {A} to {By}. 

It remains to check the statement about the endpoints. We will assume, 
again using Lemma 2.1, that if {A;} and {B,} have the same endpoint they 
agree for f near that endpoint. Then, if Bo = Ao the above construction 
gives a family of paths starting at Ao. Therefore the result for fixed initial 
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endpoint is obvious. To get fixed final endpoint one can use a similar 
argument applied to the reversed (negative) paths {A,_.}, {B+}. Finally 
consider the case of two fixed endpoints, with Aj = Bp = 1. The endpoints 
A? of the family of paths constructed above are all conjugate to A, and 
equal A; at \ = 0,1. Because Aj is generic, there is a smooth family of 
elements X, € G with Xo, X1 = I such that X,'A\X) = A). The desired 
family of paths is then {Xy TAG ee 0,1): these are positive by Lemma 2.3. 
O 


2.2 Splitting numbers 


This section summarizes what is known as Krein theory, that is the theory 
of positive paths in U. The theory is nicely described by Ekeland in [7, 8] 
where it is used to develop an index theory for closed orbits. Interestingly 
enough, this analysis is very closely related to work by Ustilovsky [20] on 
conjugate points on geodesics in Hofer geometry on the group of Hamilto- 
nian symplectomorphisms. 

It will be convenient to work over C rather than R. We extend J and w 
to C2” = R?" @C by complex linearity, so that w(v, w) = w? Jv is complex 
bilinear. Let (v,w) denote the standard Hermitian inner product on C”, 
namely 


oD 


(v,w) = & v = w(Jv, w). 


We will also use the form ( given in our notations by 

B(v, w) = —iw(v, ®) = —i(Jv, w) = —twT Jv. 
Lemma 2.5 (i) 6 is a nondegenerate Hermitian symmetric form. 
(ii) 7B(v, Jv) = (v, v). 


(iii) An element A € GL(2n, C) belongs to G if and only if it preserves B 
and ts real, t.e. 
Av = Ab, vec", 


(iv) The invariant subspaces EF), E,, corresponding to eigenvalues A, Le are 
G-orthogonal unless \ji = 1. 


(v) If v is an eigenvector with eigenvalue \ € S} of multiplicity 1 then 
B(v,v) € R—- {0}. 


Proof: ( is Hermitian because 


B(w,v) = —iw(w, v) = iw(w, v) = —iw(v, w) = B(v, w). 
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Statements (ii) and (iii) are obvious. To prove (iv), consider A € G = 
Sp(2n,R) with eigenvalues A, and corresponding eigenvectors v, w and 
observe that 


Blv,w) = B(Av, Aw) = B(w, pw) = ABB(o, w). 


The proof when v,w belong to the invariant subspace but are not eigen- 
vectors is similar (see Ekeland [8]). To prove (v), suppose A € S* is an 
eigenvalue of multiplicity 1 with eigenvector v. Then B(v,v) # 0, since 
B is non-degenerate and B(v,w) = 0 whenever w belongs to any other 
eigenspace of A. Further, G(v,v) € R by (3). oO 


It follows from (v) that simple eigenvalues A on S! may be labelled 
with a number o(A) = +1 called the splitting number chosen so that 


B(v,v) € o(A)Rt. 


For example, when n = 1, the rotation matrix 


cost —sint 
p(t) = ( sint cost ) 


(4) 222) 


corresponding to the eigenvectors et et, and it is easy to see that e” has 
splitting number +1, while e~* has splitting number —1. 

More generally, if A € S! has multiplicity > 1 we define o(A) to be 
the signature of the form § on the corresponding invariant subspace. Note 
that o(A) = —o(A). Hence the splitting number of +1 is always 0. It is 
not hard to check that the conjugacy class of an element in U is completely 
described by its spectrum together with the corresponding set of splitting 
numbers. The following result is central to our argument. 


has eigenvectors 


Lemma 2.6 (Krein) Under a positive flow simple eigenvalues on ‘o> ic- 
belled with +1 must move anti-clockwise and those labelled with —1 must 


move clockwise. 


Proof: We repeat the proof from Ekeland’s book [8] for the convenience 
of the reader. For all ¢ in some neighborhood of to, let e** be a simple 
eigenvalue of A; with corresponding eigenvector z; € C. For simplicity 
when t = to we use the subscript 0 instead of to (writing Ao, Zo etc), and 
will denote the derivative of A; at to by JP Ao, where P is positive definite. 
It is easy to check that for any x 


(Aez, Jazz) = Gala, JAZ 12) = (eax, Jaz). 
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Applying this with x = zo and differentiating at t = to, we find that 


do, . 
(JP Aozo, Jzo) = (ie 20, Jx9) 


C=) 
from which it readily follows that 


dé; (P29, Lo) 


dt t=to 7 B(£0, 20) 


Since the right hand side has the same sign as 3(zxo, 9), the result follows. 
O 


Observe, however, that not all flows whose eigenvalues move in this 
way are positive. Further, this result shows that there may not be a short 
positive path between an arbitrary pair of elements in a given conjugacy 
class, even if one allows the path to leave the conjugacy class. 

The next lemma shows that if a (simple) eigenvalue leaves the circle it 
must do so at a point with splitting number 0. This observation highlights 
the importance of the splitting number. It is not relevant to the case n = 1 
of course, but is a cornerstone of the argument in higher dimensions. 


Lemma 2.7 Let A; be any path in G and X(t) € Spec At a continuous 
family of eigenvalues such that 


Ate S*. for t< F, 
MES), for t= 7 


Suppose also that X(t) has multiplicity 1 when t > T and multiplicity 2 at 
tT. Then 
o(Ar) = 0, 


Proof: Fort >T let V, c C?" be the space generated by the eigenspaces 
Ey), Ey /Q(t): By hypothesis, this is 2-dimensional for each such t, and it 
clearly varies continuously with t. As above, is non-degenerate on each V,. 
Therefore the splitting number can be 2,0 or —2 and will be 0 if and only 
if there are non-zero v such that B(v, v) = 0. But when t > T, One) =) 
on the eigenvectors in V,. Therefore, 3 has signature 0 when t > T and so 
must also have signature 0 on Vr. oO 


3 Conjugacy classes and the positive cone 


3.1 The case n= 1 


We will write O; for the set of all elements in SL(2, R) with eigenvalues in 
R+ — {+1}, and will also divide U — {+1, +1*} into two sets O#, distin- 
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guished by the splitting number of the eigenvalue 4 in the upper half-plane 
H-1z2eE0: 32> 0); 


Thus, for elements of ORs the eigenvalue in H has splitting number +1, 
and, by considering the rotations 


re ( cost —sint i 


sint cost 
it is easy to check that 
Ove tA tA 2, Agi > Of, O7 ={A 3 iA 2, Ag, < O}. 


These sets project to the open strata in Conj. 

We now work out in detail the structure of conjugacy classes near the 
element —1. The structure at 1 is similar, but will not be so important to 
us since we are interested in paths which avoid the eigenvalue 1. 

It is not hard to check that there are 3 conjugacy classes with spectrum 
{—1}, the diagonalizable class consisting only of {—1]}, and the classes N* 
containing the nilpotent elements 


eS 


Again, the classes NV + may be distinguished by the sign of the 21-entry. 
Moreover, if we write the matrices near —1 as 
-l+¢a2 y 
aceian 
—l+2 
the elements in NV~ U {—1} U.N* are those with trace = —2 and so form 


the boundary of the cone x? + yz = 0. The interior of the cone is the set 
where x2 < —yz, and has two components 


Ve 20,250, U =e) 2) 


The rest of the space consists of points in OR with 2? > —yz. Observe that 
the labelling is chosen so that N— is in the closure of OF. (This class N— 
was labelled by — rather than + because, as we shall see, it is the place 
where positive paths leave UW.) 

The next lemma describes the structure of short positive paths in 


SL(2, R). 
Lemma 3.1 Let Atejo,1) be a short positive path in SL(2,R), and for each 
t> 0, let ' 
A= r(te, O€(0,2m), Ir®| 21, 
be an eigenvalue of Az, chosen so that it has splitting number +1, when 


O6#n. Then: 
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(i) there ise >0 such that Ay € Of for t € [0,€]; 


(ii) the function @ is increasing, and is strictly increasing except perhaps 
at the point 0, = 1; 


(iii) if Ar = —1 for any T then A; remains inU fort > T; 


(iv) if Ay enters OR it does so through a point of N~ and if it reenters U 
it must go through N+ to O7. 


(v) Ar € OF fort >0. 


Proof: We first claim that at every point N of N7~ all vectors in 
the positive tangent cone at N point into Oj. For if not, by openness 
(Lemma 2.1(i)), there would be a positive vector pointing into OF; and 
hence a positive path along which a positive eigenvalue would move clock- 
wise, in contradiction to Krein’s lemma. Since conjugation by an orthogonal 
reflection interchanges V+ and N~ and positive and negative, it follows 
that the positive tangent cone points in O7 at all points of N+. Since 
every neighbourhood of —1 contains points in NV+, it follows from open- 
ness that positive paths starting at —1l must remain in UW. Since a similar 
result holds for negative paths, there is no short positive path which starts 
with eigenvalues on R~ and ends at —1. This proves (iii) and (iv). A 
similar argument proves (i) and (v). Statement (ii) is a direct consequence 
of Lemma 2.6). i) 


Remark 3.2 (i) The above proof shows that the structure of these positive 
paths in SL(2,R) is determined by Krein’s lemma and the topology of 
the conjugacy classes. However the result may also be proved by direct 
computation. For example, if 


al (i 
w= (7 Bynes 


and P is symmetric and positive definite, then 
d tJP 
7 |=9 te) — trae) 


Hence the trace lies in (—2,2) for t > 0, which means that every positive 
path starting at N moves into U-. By invariance under conjugacy, this has 
be to true for every element in V+. 


(ii) It is implicit in the above proof that under a positive flow in SL(2, R) 
eigenvalues on R can flow in either direction. To see this explicitly, let A be 
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the diagonal matrix diag(A,1/\) (where |A| > 1) and consider the positive 


matrices 
_ 2 -1 Saye eel 
r=( 2, ae eae 


Then the derivative of the trace (and hence the flow direction of A) has 
different signs along the flows f > REN Re (LPs, 


3.2 The case n= 2 


This case contains all the complications of the general case. We will see 
that the behavior of eigenvalues along positive paths as they enter or leave 
the circle at a value in S! — {+1} is much the same as the behavior that we 
observed in the case n = 1 at the triple points +1 in Conj. When n = 2 it 
is also possible for two pairs of real eigenvalues to come together and then 
leave the real axis. However, we shall see that the positivity of the path 
imposes no essential restiction here. One indication of this is that there is 
no relevant notion of splitting number when 4 is not on the unit circle. 
We first consider the structure of elements A € Sp(4,R). As remarked 
above, the eigenvalues of A consist either of pairs \,A € S! and A,1/A € 
R — {0}, or of quadruplets d,A,1/A, 1/A where » ¢ S1UR. We will label 
these pairs and quadruplets by the element A with |A| > 1 and SA > 0. 


A generic element of G lies in one of the following open regions: 


(i) Oc, consisting of all matrices whose spectrum is a quadruplet in C — 
(RUS); 


(ii) Ou, consisting of all matrices whose eigenvalues lie in S' — {+1} and 
either all have multiplicity 1 or have multiplicity 2 and non-zero split- 
ting numbers*; 


(iii) Op, consisting of all matrices whose eigenvalues have multiplicity 1 
and lie in R — {0,+1} (divided into two components: Oz); 


(iv) Our, consisting of all matrices with 4 distinct eigenvalues, one pair 
on S! — {+1} and the other on R — Ole I 


Note that the region Oc is connected, but the others are not. 
Lemma 3.3 The codimension 1 part of the boundaries of the above regions 
are: 


(v) Bu, consisting of all non-diagonalizable matrices whose spectrum con- 
sists of a pair of conjugate points in S! — {+1} each of multiplicity 2 
and splitting number 0. 


See eS 
3By Lemma 2.7 the latter elements must be diagonalizable. 
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(vi) Br, consisting of all non-diagonalizable matrices whose spectrum is a 
pair of distinct points X,1/\ € R — {+1,0} each of multiplicity 2. 


(vii) Bur, consisting of all non-diagonalizable matrices with spectrum 


{£1,+1, A, A}, with \ eS! — {41}. 


(viii) Bry, consisting of all non-diagonalizable matrices with spectrum 
{+1, +1, ,1/A}, with X € R — {+1, 0}. 


Proof: Use Lemma 2.7. oO 


The next step is to describe the conjugacy classes of elements in the 
above parts of G. It follows easily from Lemma 2.7 that the conjugacy class 
of an element A € Oy UOr U Ou,Rr is determined by its spectrum and the 
splitting numbers of those eigenvalues lying on S!. The next lemma deals 
with the other cases. 


Lemma 3.4 (i) The conjugacy class of A € Oc¢ is entirely determined by 
its eigenvalue X € H with |A| > 1. 


(ii) There are two types of conjugacy class in By. More precisely, for each 
eigenvalue X € S'NH of multiplicity 2 and splitting number 0, there 
are two conjugacy classes of non-diagonalizable matrices, namely the 
two nilpotent classes N=. 


(iii) For each A € (—00, -1) U (1,00), there is one type of conjugacy class 
in Br. 


(iv) A matrix in Bur or Bry is conjugate to a matrix which respects 
the splitting R* = R2 @R2. Thus the conjugacy classes of By R 
are determined by  € (0,7), its splitting number, and the conjugacy 
class N~ or N+ of the casen =1. Those of Bru are determined 
by X € (—00, -1) U(1, 00) and the conjugacy class N~ or N+ of the 
case — I 


Proof: First consider an element A € Oc. By Lemma 2.5(iv), the subspace 
V spanned by the eigenvectors v, w with eigenvalues \, 1/2 is B-orthogonal 
to its complex conjugate V, because this is the span of the eigenvectors 
v,W with eigenvalues \,1/. By the same Lemma, (v, v) = B(w, w) = 0. 
Hence, because it is non-degenerate, the form B is non-zero on (v, w). Thus 
@ has zero signature on the subspaces V,V. Conversely, suppose given 
a 2-dimensional subspace V of C* which is 8-orthogonal to its complex 
conjugate V. Let B be any complex linear B-preserving automorphism of 
V and extend B to V by complex conjugation. Then B is an element of 
Sp(4, R) because it is real and preserves B. In particular, if 3 has signature 
0 on V, the subspace V is spanned by vectors v,w such that B(v,v) = 
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B(w,w) =0. Then A(v,w) 4 0 since @ is non-degenerate, and if we define 
B= B(V,A)) by setting: 


Bu=v, Bw=(1/A)w, 
B preserves 6 because 
@(Bu, Bw) = B(v, 1/w) = B(v, w). 


The corresponding element of Sp(4,R) has spectrum 4, A, 1/A,1/A. More- 
over, every element of Sp(4,R) with such spectrum has this form. In 
particular, every such element is determined by the choice of A with 
|X] > 1,SA > 0 and of the (ordered) decomposition V = Cv & Cw of 
V. Hence there is exactly one conjugacy class for each such 4. This proves 
(i). 

To prove (ii), first observe that under the given hypotheses the 
eigenspace V = E) has the same properties as in (i). Namely, Blv is a 
non-degenerate form of signature 0, and V is B-orthogonal to V. Moreover, 
the restriction of A to V can be any (-invariant complex linear map with 
eigenvalue A of multiplicity 2 and splitting number 0. Suppose that Aly 
has an eigenvector v with G(v,v) # 0. Then there is a 3-orthogonal vector 
w such that B(w,w) = —B(v,v), and because 


B(w,w) = B(Aw, Aw) = B(Aw + pv, Aw + po) = B(w, w) + [pl?B(v,») 


we must have p = 0. Thus A is diagonalizable (over C) in this case. Gg 
no such v exists, we may choose a basis v,w for V such that 6(v,v) = 
B(w,w) = 0 and so that Av = Av, Aw = \w + pv, for some p # 0. If we 
also set G(v,w) = 1, it is easy to see that B(Aw, Aw) = B(w,w) only if Az 
is real. Given v, our choices have determined w up to a transformation of 
the form wt> w’ = w+ «Kv, where « € R since 6(w’,w’) = 0. It is easy to 
check that if we make this alteration in w then p does not change. On the 
other hand, if we rescale v, w replacing them by v/k, & w then p changes by 
the positive factor |«|?. Hence we may assume that w = +A. This shows 
that Aly is conjugate to exactly one of the matrices 


rd 0 ee eM nef» 0 
Dill e sel al N =i(4 v 


To prove (iii), observe that the eigenspace W of (where |A| > 1) is 
Lagrangian, and by suitably conjugating A by an element of G we may 
suppose that the other eigenspace is JW. It then follows that A has the 
foun AC Ge (Ga) J forsome@ = SL(2,R) with trace = 2. In fact, 
any linear map of the form B® J™(B-)T J, where B € GL(2,R), is in 
G. It follows easily that there are two conjugacy classes, one in which C is 
diagonalizable, and one in which C is not. 
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(iv) Finally, consider a matrix A in By.z with a pair of eigenvalues X, A, 
with A € S'NH where H is the upper half plane, and the eigenvalue say —1, 
with multiplicity 2. Assume say that the splitting number of ) is positive. 
Let V be the eigenspace generated by the eigenvectors v, w corresponding 
to the eigenvalues 1,4, and V’ the invariant subspace associated to the 
double eigenvalue —1. By Lemma 2.5, v is G-orthogonal to V’ and w, 
and #(v,v) > 0 by assumption. Similarly, w is 6-orthogonal to V’ and 
v, and G(w,w) < 0. Thus the restriction of @ to V’ is a also a non- 
degenerate symmetric Hermitian form with zero signature. Because A is 
non-diagonalizable, its restriction to V’ is non-diagonalizable. Note that 
both eigenspaces V and V’ are invariant under conjugation, and therefore 
that the restrictions of A to each subspace is real. This shows that we can 
identify V @V’ with C(R*) @ C(R?) by a linear map f which respects the 
factors, preserves 6 and is real, and such that fAf—! is the direct sum of 
a rotation on the first factor and of an element in N= on the other factor. 

The proof is similar in the other cases. Oo 


We next investigate the relationship between the positive cone field and 
the projection 7 : G — Conj. Krein’s Lemma 2.6 shows that the movement 
of eigenvalues along positive paths in Oy and Ou,R is constrained. However, 
this is not so for the other open strata. 


Lemma 3.5 The projection t : G — Conj maps the positive cone P, at 
A € Oc onto the tangent space to Conj at n(A). A similar statement is 
true for AE Or. 


Proof: Recall from Lemma 3.4 that an element A of Oc defines a unique 
splitting V@ V, where 3 lv has signature 0 and where Alv has eigenvalues 
X,1/A for some \ € {Sz > 0}. Moreover, we may choose the eigenvectors 
v,w for A, 1/X respectively so that 


B(v,v) = B(w,w) = 0, B(v,w) = 1. 


Therefore, if we fix such v, w we get a unique representative of each conju- 
gacy class by varying X. 

Most such splittings V @ V are not J-invariant, and so it is hard to 
describe the positive flows at A. However, because the positive cone field 
on G is invariant under conjugation (see Lemma 2.3), we only need to 
prove this statement for one representative of each conjugacy class, and so 
we choose one where this splitting is J-invariant. For example, if we set 
aly V2(1, 0,7, 0) and w = —Jv, all the required identities are satisfied. 
Moreover, the transformation X which takes the standard basis to v, w, 0, w 
is unitary. Therefore, any linear automorphism B of V which is represented 
by a positive definite matrix with respect to the basis v,w extends by 
complex conjugation to a linear automorphism B @ B of C4 which is also 
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positive definite with respect to the usual basis. This means that there are 
positive tangent vectors to G at A which preserve V and restrict to JB 
there. We are now essentially reduced to the 2-dimensional case. Just as 
in Lemma 3.1, if B is a real positive definite matrix with negative 21-entry 
then the eigenvalue 4 moves along a ray towards S 1 and if its 21-entry is 
positive 4 moves along this ray away from S 1 Since 7 maps the positive 
cone onto an open convex cone, this means that the image has to be the 
whole tangent space to Conj. 

The argument for A € Og is similar but easier. oO 


The next step is to study generic positive paths through the various 
parts of the codimension 1 boundary components. We will see that Krein’s 
lemma forces the behavior of positive paths at the boundary By to mimic 
that that near —1 in the case n = 1, while the boundary Bg is essentially 
not seen by positive paths. Our results may be summarized in the following 
proposition. 


Proposition 3.6 (i) If AE Ny C Bu, then m maps the positive cone Pa 
into the set of vectors at n(A) which point into 7(Oc). Similarly, af 
AEN, , then 7.(Pa) consists of vectors pointing into mu). 


(ii) If A € Br then 7,(Pa) contains vectors which point into 7(Oc) as 
well as vectors pointing into n(OrR). 


(iii) If A € Bur UBry then A is conjugate to an element which preserves 
the splitting R* = R? @ R?, and 1, takes the subcone in Pa formed 
by vectors which preserve this splitting onto the whole image m.(Pa). 
In other words, positive paths through By,R and Bry behave just like 
paths in the product SL(2,R) x SL(2, R). 


Proof: (i) Suppose first that A € Nx. We claim that there is a neigh- 
bourhood of A € G whose intersection with U consists of elements which 
have an eigenvalue X’ with splitting number +1 which is close to 4 and to 
the right of it, i.e with arg )’ < argi. To see this, observe first that by 
conjugacy invariance it suffices to prove this for a neighbourhood of A in 
the subgroup 
Gy ={A'eG: A(V) =V}. 

If A’ € Gy, then A’|y may be written as \’Y where } is close to A, and Y 
has determinant 1 and preserves #3. If 


a b 
v=(¢ a) 
with respect to the basis v,w, then we must have ac € R, bd € R and 
ad — cb = 1 in order to preserve G and trY = 2. This implies that Y 
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must be in SL(2,R). It is also close to the element +Aly € N~ = N72). 
Therefore, the result follows from the corresponding result in the case n = 1: 
see the discussion just before Lemma 3.1. 

Since eigenvalues with positive splitting number flow anticlockwise un- 
der a positive path (by Krein), every positive path through A € Ny must 
have all eigenvalues off S! for t > T and so must enter Oc. Similarly, 
positive paths through any point of N. ce flow into YU. As in the case when 
A = —1, this implies that positive paths through points of D, remain in U. 


(ii) Suppose first that A in the closure Br of Br is diagonalizable, say 
A = diag(\, 1/4, A,1/A). Then, as in Remark 3.2, there is a positive flow 
starting at A which keeps A diagonalizable and in Bp, while decreasing |)]. 
Since any sufficiently C!-small perturbation of this flow is still positive, we 
can clearly flow positively from A into both regions Or and Oc. 

Next observe that if we fix \ € R and consider only those elements of 
Br with fixed A-eigenspace W and 1/)-eigenspace W’, then these form a 
cone C(\) whose vertex is the diagonalizable element D). (The structure 
here is just like that near —11 which was discussed in §3.1.) Therefore, if v is 
a positive vector at D) which points into Oc, there is a nearby vector v’ at a 
nearby (non-diagonalizable) point A which also points into Oc. Moreover, 
we may assume that v’ is positive by the openness of the positive cone. 
Therefore one can move from A into Og, and similarly into Or. Since 
there is only one conjugacy class of such A for each X, this completes the 
proof. 


(iii) As shown in Lemma 3.1, the behavior of positive paths at the two 
conjugacy classes —I* is dictated by Krein’s lemma and the topology of 
SL(2,R). A similar argument applies here. QO 


Remark 3.7 (i) One can construct explicit paths from A € Br into Oc 
and Op as follows. Note that the matrices A € Br all satisfy the relation 
oF 


2 
Dae 


02> 


where o,; denotes the jth symmetric function of the eigenvalues. Similarly, 
one can easily check that the matrices in Og satisfy the condition 02> soy, 


2 
while those in Og satisfy a2 < = +2. Therefore, to see where the positive 
path A; = (1+tJP+...)A goes as t increases from 0 we just have to 
compare the derivatives 
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For example, if we take 


r 0 0 0 1 0 0 O 
(= = 0 «@ Cw SP) 
A A = 
= 55 OA On ale P =? ier. | 
ora) 0, Dial 
then 
A— p— 2 es 2tA —Bat 
(1 +2JPA)-—pii= et a 
-& —3t th $—p-—2at 
Thus : 
ey z ae 
a 2 (5 +A + 53) 
while 9 
Therefore 
O10, 1 ae 1/2 —3/2\2 
Oy — 5 = -2a(d+ 53 — x) = ~2a”’ = Nariel 


and the path goes into Oc ifa <0 and into Og if a > 0. 


(ii) Observe also that we did not show that 7 takes the positive cone at a 
point of Br onto the full tangent space of (Oc) = H. However, because 
of Lemma 2.1(iii), all that matters to us is that movement between these 
zones is possible. 


4 Proof of the main results 


4.1 The casen<2 
For the convenience of the reader we now restate Theorem 1.2. 
Theorem 4.1 Let n < 2. 


(i) An element of G— S, is the endpoint of a short positive path (from 1) 
if and only if it has an even number of real eigenvalues with > 1. 


(ii) There is a positive path between any two elements A,B € G. Moreover, 
any short positive path from 1 may be extended to a short positive path 
with endpoint in U. 
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(iii) The space of short positive paths (from 1) with endpoint in U is path- 
connected. 


Proof: (i) There are two ways eigenvalues can reach the positive real axis 
Rt. Either a pair of eigenvalues moves from S$! through +1 to a pair on 
R*, or a quadruplet moves from C —R to R. The first scenario cannot 
happen along a short path: for the only time a short path has an eigenvalue 
+1 is at time t = 0 and then, by Lemma 3.1, the eigenvalues have to move 
into S'. In the second case, one gets an even number of eigenvalues on 
(ites). 

(ii) Using Proposition 3.6 it is not hard to see that there is a positive path 
in Conj from m(A) to 1 and from 1 to 7(B). The result now follows from 
Lemma 2.3. The second statement is proved similarly. 


(iii) We have to show that any two short positive paths {Az}, {B:} begin- 
ning at il and ending in UY can be joined by a homotopy of short positive 
paths beginning at Il and ending in U/. We may clearly assume that both 
paths are generic. (Note however that the homotopy of paths may have to 
go through codimension 2 strata.) Moreover, by Lemmas 2.1 and 2.6 we 
may assume that there is 6 > 0 such that A; = B, €U for t € [0,6]. There- 
fore, it will suffice to consider the case when the second path {Bz} is (a 
reparametrization of) A,,0 <t < 6. Thus we have to show how to “shrink” 
the path {A;} down to its initial segment in U/, keeping its endpoint in UW. 

We do this by constructing a homotopy 4 in Conj between the projec- 
tions of the two paths which at each time obeys the restrictions stated in 
Proposition 3.6. Indeed, such a homotopy means the existence of a homo- 
topy 7° of short positive paths in G from 1 and ending in YU which satisfies: 
4) = A, for all t, 7 = 1 and 4 € U for all yu, and 7(7}) = m(B,). By 
Proposition 2.4, there is a homotopy of paths (with starting points equal to 
1 and endpoints free in the conjugacy class of B,) between {4}} and {B;}. 
The composition of these two homotopies is the desired path from {A;} to 
{Bz}. 

We construct the homotopy 7“ in Conj by means of a smooth map 
r:mt+r* from the interval [0,1] to the space of short positive paths in 
Conj as follows.4 Suppose that we have found r which satisfy: 

(i) r#(u) = n(A,) and r*(1) € mW); 
(ii) the paths r# are constant (with respect to t) for pw near 0. 
Then it is not hard to check that we may take 74 to be: 


nip — 7 (Adenia 
a r4(t) if ¢> p. 


We will construct the r# backwards, starting at up = 1. Intuitively, 
r® should be the simplest path from m™(A,,) to 7(U) (ie. it should cross 


“As will become clear, we are only interested in r“(t) on the time interval t € [u, 1). 
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the stratum Conj, as few times as possible), and we should think of the 
homotopy y“ as shrinking away the kinks in {A;} as p decreases. To be 
more precise, let us define the complexity c(y) of a generic path y = {C;} 
to be the number of times that 7(C;) crosses Conj,. Then we will construct 
the r# so that c(y") does not increase as j decreases. Moreover it decreases 
by 2 every time that A, moves (as pz decreases) from Og to Oc, or from 
Ou,r to Oy or from Oc to Oy. 

To this end we assume that, except near places where 7(A,,) € Conj,, 
the paths r’ have the following form: 


e if A, €U then r* is constant; 

e if A, € Oc is labelled with \ = se®® then r, goes down the ray s‘e® until 
it meets the circle in 7(N;{,) and enters ™(Y/); 

e if A, € Og then r* moves the two eigenvalue pairs together, pushes them 
into 7(Oc) and then follows the previous route to m(U); 

e if Ay € Ou,r then r@ fixes the eigenvalues in S' and moves the real ones 
through 7(N7*,) to S?. 


We now describe how to extend r over each type of crossing. While 
doing this, there will be times p at which we will want to splice different 
positive paths r# together. More precisely, at these 4; we will have two 
different choices for the path r#i from A,, to U. But these choices will 
be homotopic (by a homotopy which respects the endpoint conditions) and 
so, if we reparametrize the path A; so that it stops at ¢ = pj for a little 
while, we may homotop from one choice to the other. To be more precise, 
we choose a nondecreasing function p : [0,1] — [0, 1] which is bijective over 
all points except the pj and is such that p~!(;) is an interval, and then 
change the relation between r*, 4" and A; by requiring that 


e r#(t) is defined for t € [p(u), 1] and r¥(p(p)) = Apu), 
o y#(t) = Ant S ou). 

First let us consider how to handle crossings of the stratum By. Such 
crossings take place either at N. A or Ny . However, because positive paths 
starting at NV. Rs point into U there is no problem extending r smoothly 
over this type of crossing. Observe that at this crossing the complexity 
remains unchanged. The problem comes when A, € Ny. To deal with 
this, first let C, be a positive path which starts in U, crosses Ny at time 
yu, goes a little into Oc and then crosses back into U through VN. We may 
choose this path so that it is homotopic through positive paths with fixed 
endpoints to a positive path lying entirely in U. (In fact, we could start 
with a suitable path in U/ and then perturb it.) By Lemma 2.1, we may 
suppose that (Ar) = m(C;,) for t in some interval (44 — €,4+€) and so 
(using the splicing technique described above) we may choose r so that, for 
some p! € (1, 4+€), we have r* (t) = 7(C;) for t > p’. We now use this 
formula to extend r over the interval (u — ¢, p’). Then r#~* = {C;} starts 
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and ends in U and goes only a little way outside U/. We now reparametrize 
{A;} stopping it at time t =  — € so that there is time first to homotop 
r#~© = {C;} with fixed endpoints to a path in U and then to shrink it, fixing 
its first endpoint A,_-, to a constant path. This completes the crossing. 
Note that the complexity does decrease by 2. 

A little thought shows that the same technique may be used to deal 
with crossings of the other boundaries Bur, Bry and Br. For example, 
if as decreases A,, passes from Oc to Or, one easily extends r but does 
not change the complexity of y“. On the other hand, if A, passes from Or 
to Oc one needs to take more trouble in extending r but in exchange one 
decreases the complexity. oO 


4.2 The case n>2 


As previously explained, the eigenvalues of an element of Sp(2n,R) form 
complex quadruplets or pairs on S$! or R. It is easy to check that the only 
singularities (or bifurcations) encountered by a generic 1-dimensional path 
Y = Atejo,1) are those in which 


(a) a pair of eigenvalues on S1 or R become equal to = I 


(b) two pairs of eigenvalues coincide on S! or R and then move into C (or 
conversely). 


In particular, two quadruplets or three pairs do not coincide generically. 
Therefore, for each t, the space R2” decomposes into a sum of 2- and 4- 
dimensional eigenspaces E\(t) @... @ E,(t). Moreover, the interval [0, 1] 
may be divided into a finite number of subintervals over which this decom- 
position varies smoothly with t. (The type of the decomposition may be 
different in the different pieces.) We claim that within each such piece the 
eigenvalue flow is just the same as it would be if the decomposition were 
fixed rather than varying. The reason for this is that all the restrictions on 
the eigenvalue flow are forced by Krein’s lemma, which is valid for arbitrary 
variation of decomposition, together with topological data concerning Conj 
(i.e. topological information on the way the types of conjugacy classes fit 
together.) With these remarks it is not hard to adapt all the above argu- 
ments to the general case. In particular, the proof of (iii) is not essentially 
more difficult when n > 2 since in this case we reduce the complexity of the 
path as we procede with the homotopy. It is important to note that there 
still are essentially unique ways of choosing the paths r from the different 
components of the top stratum Co into 1(U). (“Essentially unique” means 
that the set of choices is connected.) For example, if A” were on a stratum 
with 3 eigenvalue pairs on R~, then one might combine two to make a 
quadruplet in C which then moves down a ray to S? leaving one pair to 
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move through NV+, to S', or one might move all 3 pairs directly down R to 
S). But these paths are homotopic (through positive paths with endpoints 
on 7(U)) and so it is immaterial which choice we make. Oo 


5 Positive paths in Hamiltonian systems and 
in Hofer’s geometry 


We present here a brief outline of the various ways in which positive paths 
intervene in the stability theory of Hamiltonian systems, and in Hofer’s 
geometry. Actually, they are also a crucial ingredient of the theory of 
closed geodesics as developed by Bott in a series of papers ((3, 4, 5]). But 
since the application of positive paths to closed geodesics (and in particular 
to the computation of the index of iterates of a given closed geodesic) can 
also be presented within the framework of positive Hamiltonian systems via 
the geodesic flow (where Ekeland’s formula appears as a generalization of 
Bott’s iteration formula), this application is, at least theoretically, reducible 
to the following one. 


5.1 Periodic Hamiltonian systems 


Many Hamiltonian systems are given as periodic perturbations of au- 
tonomous flows. When both the autonomous Hamiltonian and the per- 
turbation are quadratic maps Q:, the fundamental solution of the periodic 
system 


a(t) _JIVxOx(2(t)), where Q = Q41, 
a(0) = & 


is a path of matrices A; € G as in 81. (In other words, the trajectory x(t) 
which starts at £ is A;().) By the periodicity of the generating Hamil- 
tonian, it is clear that Ax+t = A, A* for all integers k and real numbers 
t € [0, 1). 


Il 


Definition 5.1 The above periodic system is stable if all solutions x(t) 
remain bounded for all times (in other words, it is stable if there is a 
constant C such that || A;|| < C for all t > 0). A matrix A is stable if there 
is C such that ||A*|| < C for all positive integers k. 


Clearly, the above system is stable if and only if A = Aj is stable. But 
it is easily seen that a matrix A is stable if and only if all its eigenvalues lie 
on the unit circle. Thus the stability of the periodic Hamiltonian system is 
determined by the spectrum of its time-1 flow A. 
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Definition 5.2 The Hamiltonian system is strongly stable if any C?- 
small periodic (and quadratic) perturbation of it remains stable. A matrix 
A is strongly stable if it has a neighbourhood which consists only of stable 
matrices. 


Since a C?-small perturbation Q/ of the Hamiltonian Q, leads to a flow 
te[o,1] Which is C'-close (as a path) to the unperturbed flow, the time-1 
map A’ = A’(1) is C°-close to the time-1 map A = A(1). Conversely any 
matrix A’ close enough to A in the symplectic group is the time-1 map of 
a C?-small quadratic perturbation of Q+¢j0,1). This means that a periodic 
Hamiltonian system is strongly stable if and only if the time-1 map A is 
strongly stable. Hence such a system is strongly stable exactly when its 
time-1 flow belongs to the interior of the set of symplectic matrices with 
spectrum in S!. But we have seen in the previous sections that this interior 
consists of all matrices with spectrum in S$! — {+1} and maximal splitting 
numbers. (More precisely, at each multiple eigenvalue the absolute value of 
the splitting number must equal the multiplicity of the eigenvalue.) This is 
the Stability Theorem due to Krein and Gelfand-Lidskii. 

Now consider the case n = 2 and assume that the stable periodic Hamil- 
tonian Q; is negative and that its flow {A;} is short (that is the index of 
{At} is zero). Further, let NU be the union of all open strata for which 
none of the eigenvalues lie on $1, and let e,4 (s) be the number of times that 
the path {A,} enters NU and comes back to Oy during the time interval 
(0, s]. 


Lemma 5.3 For every short positive path Atco, beginning at 1, e A= 
We 


Proof: Because NU and U are open, it suffices to prove this when the 
path {Az} is generic. If e4(1) = 0 there is nothing to prove. So suppose 
that e4(s) = 1 for some s < 1. Then Proposition 3.6 implies that on 
the interval [0,s] either A; moves from Og to Oy passing through some 
conjugacy class N, Re at some time tp < s or A; moves from OR into Oy via 
matrices with all eigenvalues on (—co,-1)U S!. In either case, it follows 
from Krein’s lemma 2.6 that the first eigenvalue of A, that one encounters 
when traversing S$! anticlockwise from 1 has negative splitting number. 
Therefore, by Lemma 2.7, the eigenvalues of A;,t > s, can only leave $1 
after a pair has crossed +1. Oo 


More generally, this argument shows that for a stable positive path 
Ate{o,1], Which is not necessarily short or generic, e4(1) < 7+1, where 
t =7,(1) is the Conley-Zehnder index of Atejo,1)- (In this context, i A(1) is 
defined to be the number of times t > 0 that a generic positive perturbation 
{A;} of {A:} (with A = Ao = 1) crosses the set 5S, = {A: det(A—1) = 
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0}. Observe that because {A{} is positive each such crossing occurs with 
positive orientation.) 

Combining this with Ekeland’s version of Bott’s iterated index formula 
(see [8]), we get: 


Proposition 5.4 Let A:>o be a stable positive path generated by a 1- 
periodic Hamiltonian in R*. Then there is a positive real number a such 
that for any time t > 0: 


eat is alta! 
where {t] is the greatest integer less than or equal to t. 


Thus some aspects of the qualitative behavior of such paths are linearly 
controlled by the Conley-Zehnder index. Similarly, the number of times 
that such a path A; leaves Oy, enters any other open stratum and comes 
back to Oy during the time interval (0, t], is bounded above by 


SGA 1) < 3(a {t] ia(1) +1). 


As a last application, noted by Krein in [10], Chap VI, consider the 
system <(t) = —pJVxQ:(x(t)), Qe+1 = Q:, with a parameter p € R. 
Then: 


Proposition 5.5 (Krein) Jf each Q; 1s negative definite, there is 9 > 0 
such that the periodic Hamiltonian system 


£(t) = —pJIVx Q(2(t)) 
is strongly stable for all 0 < p < po- 


Actually, the value pio is the smallest ys such that the time-1 flow A, 
of the system has at least one eigenvalue equal to —1. Hence each time-1 
flow A, for » < 0 must be strongly stable. 


5.2 Hofer’s geometry 


Hofer’s geometry is the geometry of the group Ham(M) of all smooth 
Hamiltonian diffeomorphisms, generated by Hamiltonians Hyejo1) with 
compact support, of a given symplectic manifold M, endowed with the 


bi-invariant norm 
1 
=F — min H,) dt 
loll = inf i (max H; — min H:) 


where the infimum is taken over all smooth compactly supported Hamilto- 
nians Hy¢jo,1] On M whose time-1 flows are equal to ¢. This norm defines 
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a Finslerian metric (which is L© with respect to space and L! with re- 
spect to time) on the infinite dimensional Fréchet manifold Ham(M). In 
[15] and [2], it is shown that a path ~, generated by a Hamiltonian H; is 
a geodesic in this geometry exactly when there are two points p,P € M 
such that p is a minimum of H; and P is a maximum of H; for all t. We 
then showed in [16] that the path Yre{0,1] is a stable geodesic (that is a 
local minimum of the length functional) if the linearized flows at p and 
P have Conley-Zehnder index equal to 0. (See also [20].) Note that the 
linear flow at p is positive while the one at P is negative. The condition 
on the Conley-Zehnder index means that both flows are short. The proof 
of the above sufficient condition for the stability of geodesics in [16] relies 
on the application of holomorphic methods which reduce the problem to 
the purely topological Main Theorem 1.2. Hence the result of the present 
paper can be considered as the topological part of the proof of the stability 
criterion in Hofer geometry. We refer the reader to [16] for the equivalence 
between this stability criterion and the local squeezability of compact sets 
in cylinders. 


References 


[1] V.I. Arnold and A. Givental, Symplectic Topology, in Dynamical 
Systems, vol IV, Springer-Verlag, Berlin, 1990. 


[2] M. Bialy and L. Polterovich, Geodesics of Hofer’s metric on the 
group of Hamiltonian diffeomorphisms, Duke J. Math. 76 (1994), 
213-292. 


(3 


ey 


R. Bott, Non-degenerate critical manifolds, Ann. Math. 60 (1954), 
248-261. 


[4] R. Bott, On the iteration of closed geodesics and Sturm intersection 
theory, Comm. Pure Appl. Math 9 (1956), 173-206. 


[5] R. Bott, The periodicity theorem for classical] groups, Ann. Math. 
70 (1959), 179-203. 


[6] C. Conley and E. Zehnder, Morse type index theory for flows and 
periodic solutions for Hamiltonian equations, Comm. Pure Appl. 
Math 27 (1984), 211-253. 


[7] I. Ekeland, An index theory for periodic solutions of convex Hamil- 
tonian systems, Proc. Symp. Pure Math 45 (1986), 395-423. 


[8] I. Ekeland, Convexity Methods in Hamiltonian Mechanics, Ergeb- 
nisse Math 19, Springer-Verlag Berlin, 1989. 


F. Lalonde, D. McDuff 387 


[9] I. Gelfand and V. Lidskii, On the structure of the regions of stability 
of linear canonical systems of differential equations with periodic 
coefficients, Uspekhi Math. Nauk. USSR 10 (1955), 3 — 40 (AMS 
translation 8 (1958), 143-181). 


[10] I.C. Gohberg and M.G. Krein, Theory and Applications of Volterra 
Operators in Hilbert Space, Transl. of Math. Monogr. vol. 24, Amer- 
ican Mathematical Society, 1970. 


[11] K. Grasse and H.J. Sussmann, Global controllability by nice con- 
trols, Nonlinear Controllability and Oprimal Control, ed. H. J. Suss- 
mann, Dekker, New York, 1990, pp. 38-81. 


[12] M. Gromov, Carnot-Caratheodory geometry, preprint THES, 1993. 


[13] N. Hingston, Perambulations in the symplectic group, in prepara- 
tion. 


[14] W. Klingenberg, Lectures on Closed Geodesics, Springer-Verlag, 
Berlin, 1981. 


[15] F. Lalonde and D. McDuff, Hofer’s L™-geometry: energy and sta- 
bility of Hamiltonian flows parts I and HU, to appear in Invent. 
Math.. 


[16] F. Lalonde and D. McDuff, Local non-squeezing theorems and sta- 
bility, Geom. Funct. Anal. 5 (1995), 364-386. 


[17] C. Lobry, Controllability of nonlinear systems on compact mani- 
folds, SIAM Journ on Control 12 (1974), 1-4. 


[18] J.W. Robbin and D.A. Salamon, The Maslov index for paths, 
Topology, 32 (1993), 827-44. 


[19] H.J. Sussmann, Reachability by means of nice controls, Proc 26th 
IEEE conference on Decision and Control, Los Angeles, (1987), 
1368-1373. 


[20] I. Ustilovsky, Conjugate points on geodesics of Hofer’s metric, 
preprint, Tel Aviv, 1994. 


Francois Lalonde Dusa McDuff 

Univ. de Québec & Montréal SUNY 

Dept. Math. Dept. Math. 

Montreal PQ H3C 3P8, Canada Stony Brook, NY 11794-3651 
flalonde@math.uqam.ca dusa@math.sunysb.edu 


August 25, 1995 


Invariants of submanifolds in Euclidean space 


V.D. Sedykh 


ABSTRACT. The topology of the set of singular support hyperplanes and 
hyperspheres to a smooth submanifold in Euclidean space is studied. As a 
corollary, some relations between differential-geometric characteristics of a 
manifold are obtained. In particular, if a simple closed embedded generic 
curve in a plane has C global vertices (where the curvature circles are 
support circles to the curve) and T support circles touching the curve at 
three points, then C — T = 4. Similar invariants are also obtained for 
submanifolds in higher-dimensional spaces. 


INTRODUCTION 


A tangent hyperplane or hypersphere to a smooth submanifold in Eu- 
clidean space is called a support one if the manifold lies on one side of it. A 
nonsingular support hyperplane (hypersphere) is tangent to the manifold 
at just one point and has at this point first-order tangency with the germ of 
any curve on the manifold. All other support hyperplanes and hyperspheres 
are singular. 

In the paper, we compute the homology groups of the set of singu- 
lar support hyperplanes and hyperspheres to a closed connected manifold 
which is not contained in any hyperplane and in any hypersphere in the 
ambient space. The Euler number of the set of their tangent elements to the 
manifold is computed as well. As a corollary, we obtain numerical relations 
between different classes of singular support hyperplanes and hyperspheres 
to a submanifold in Euclidean space. 

Take, for example, a simple closed embedded generic curve in a plane. 
A vertex of a curve (the local extremum point of its curvature) is said to be 
global if the curvature circle at this vertex is a support circle to the curve. 
Let C be the number of global vertices of a curve and T' be the number of 
support circles tangent to the curve at three points. Then 


C-T=4. 


Support circles lying inside (outside) of a curve are called internal (ex- 
ternal). Vertices of a curve where the curvature circles are internal (ex- 
ternal) support circles are said to be internal (external). Suppose that a 
curve has C internal and C2 external vertices, T; internal and T> external 
support circles tangent to the curve at three points. Then 
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Cn Cy 


The first from the above relations implies the classical 4-vertex theorem 
(in the generic case): any smooth closed embedded plane curve has at least 
four vertices. Indeed, the number of all vertices is greater than or equal 
to the number C of global vertices of the curve, and C = T+ 4 eae 
We generalize this relation for the case of generic submanifolds in higher- 
dimensional spaces. 


1. SINGULAR SUPPORT HYPERPLANES AND HYPERSPHERES 


Let M be a C°-smooth k-dimensional submanifold (k > 0) in Eu- 
clidean space R”. Consider affine hyperplanes and hyperspheres (i.e. stan- 
dard (n — 1)-dimensional spheres) in R” tangent to M. 


Definition. A tangent hyperplane (hypersphere) to a manifold M is said 
to be a support one if M lies on one side of this hyperplane (hypersphere). 


Definition. A support hyperplane (hypersphere) 7 to a manifold M is 
called nonsingular if 

1) 7 is tangent to M at just one point P, and 

2) any smooth curve lying on M and passing through P has at this 
point first-order tangency with 7. 

All other support hyperplanes and hyperspheres to M are said to be 
singular. 


Definition. A manifold M is called nonflat if it is not contained in any 
hyperplane and in any hypersphere in R”. 


Denote by © the set of all singular support hyperplanes and hyper- 
spheres to a manifold M. If M is closed then ¥ is a compact subset in the 
smooth (n + 1)-dimensional manifold of all hyperplanes and hyperspheres 
in R”. 

Theorem 1. Let M be a smooth closed connected nonflat k-dimensional 
submanifold in R". Then the homology groups and the Euler number of the 
set & are given by the following formulas: 


Z iffk<n-1, 


eel (Ml, Za Ore th 
Ht B.0) 2 ifm>n-1, 


(1) x(Z) = x(S") — (-1)"-*y(M). 
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In particular, for k = n — 1 the set © has two connected components 
and H,(X,Z) & Hn(M,Z), 0 <m<n-1. All hyperspheres from a given 
component lie on one side of M. 

The detailed proof of Theorem 1 will be published in [S3]. We consider 
the stereographic image of the manifold M on the unit sphere in Reet 
and apply to it the results of [S1] ([S2]) on topology of the set of singular 
support hyperplanes to a convex submanifold (lying on the boundary of its 
convex hull) of codimension greater than 1 in an affine space. 


2. TANGENT ELEMENTS OF SINGULAR SUPPORT HYPERPLANES 
AND HYPERSPHERES 


Let us consider a hyperplane (hypersphere) 7 in R” tangent to a sub- 
manifold M at a point P. 


Definition. The pair (7, P) is called a tangent element of m to M at P. 


The set L of tangent elements of all tangent hyperplanes and hyper- 
spheres to a k-dimensional manifold M isa smooth n-dimensional manifold. 
The mapping that assigns to a tangent element the point of tangency defines 
a smooth fibration of L over M with fiber RP”~* \ {+}. 

Denote by 2 the set of tangent elements of singular support hyperplanes 
and hyperspheres to M. If M is closed then 2 is a compact subset in LD. 


Theorem 2. Let M be a smooth closed connected nonflat k-dimensional 
submanifold in R". Then the mapping 


Q— M, (x1, P)-— P 


defines a C°-fibration of Q over M with fiber S°-*-1. In particular, the 
Euler number of the set Q is given by the following formula: 


(2) (Oy — om al) 


Fork =n—1, the set Q consists of two connected components each of which 
is homeomorphic to M. 


The proof of this statement will be given in (S3]. It is also based on 
the results of (S1]. 


3. SUPPORT A,,---Ay,,-PLANES AND SPHERES 


We consider the space of all smooth embeddings of a closed k- 
dimensional (k > 0) manifold M into R” equipped with the C™-topology. 
The submanifolds corresponding to the embeddings from a certain open 
everywhere dense subset in this space are said to be generic. Generic sub- 
manifolds in R™ are nonflat. 

Let us take a sequence of odd natural numbers [41 < ... < Mm. 
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Definition. A support hyperplane (hypersphere) 7 to a k-dimensional sub- 
manifold M in R” is called A,,...A,,,-plane (sphere) if 

1) m is tangent to M at m points A,,, ..,A,,, that are vertices of a 
(m — 1)-dimensional simplex, and 

2) the germ at A,,,i = 1,...,m, of the restriction to M of any smooth 
function in R” equal to 0 on 7, nonnegative on M and having noncritical 
value at A, is given by the formula 


th ee tee ee 


in suitable local coordinates t1,...,tm on M. 
The number d = j41+...+f4m is called the degree of a support A,,...Ay,,- 
plane (sphere). 


Let M be a smooth closed k-dimensional generic submanifold in R” 
where either k = 1 or n < 6. Then the set A of all support hyperplanes and 
hyperspheres to M consists of support Ay, ---Ay,,-planes of degree d < n 
and support A,,,...A,,,-Spheres of degree d <n+ 1 (see [A], [Z]). 

The set A((11,..., fm) of all support A,,..-A,,,-planes and spheres to a 
manifold M is a smooth submanifold of codimension d = fy +... +ftm in the 
manifold of all hyperplanes and hyperspheres in R”. The partition of the 
set A into the connected components (strata) of the manifolds A iui) 
for all possible sequences 4; < ... < fim is a minimal finite Whitney C™- 
stratification. 

This stratification determines a minimal finite Whitney C®- 
stratification of the set © of singular support hyperplanes and hyperspheres 
to the manifold M. In turn, this induces a finite Whitney C-stratification 
of the set Q of tangent elements of singular support hyperplanes and hyper- 
spheres to M. Namely, a stratum o in ¥ determines strata in Q consisting 
of the tangent elements of those hyperplanes and hyperspheres that belong 
tog. 

Let x(/41,..-;{m) be the Euler number of the manifold va ieee len) 
Denote by N(d) the set of all sequences of odd natural numbers jp < ... < 
Mm such that 44 +... + Um =d. Then the equalities (1) and (2) imply 
Corollary 1. For any smooth closed connected k-dimensional generic sub- 
manifold in R", where either k =1 orn< 6, the following relations hold: 

ar oaalt 
Dot xt ss tm) = eS") — (PEM), 
a2 (t41,.--544m)EN(d) 
n+1 
DD Si aren ee — x(S*-*1)y(M). 
d=2 (441.++544m)EN(d) 
In particular, if n is even and k is odd then 
n+1 i 
(3) >5(-1) Ss (m — an 1)x(H1, +) tm) = 0 + 2. 
d=2 


(141,.--sftm)EN(d) 
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From Theorems 1 and 2, we have also 


Corollary 2. Let M be a smooth closed connected generic hypersurface in 
R", where 2 <n <6. Then the set © (of singular support hyperplanes and 
hyperspheres to M) has two connected components ©; and Xp. The Euler 
number of each component is given by 


(4) 


1 n+1 
xB) = 5 | x(M)+ (19 (m= 2) x(a Hm) | 
d=3 (11 y+++y/4m) EN (d) 


where Xi(111,-.-; 4m) is the Euler number of the manifold 5;N.A(p1, .--; Um); 
f= LD 


4, APPLICATION TO PLANE CURVES 


Let M be a simple closed embedded generic curve in a plane. Then the 
relation (3) implies 
C-T=4, 


where C' = (3) is the number of global vertices of M (where the curvature 
circles are support circles to the curve) and T' = y(1, 1,1) is the number of 
support circles tangent to M at three points. 

According to Theorem 1, the set of singular support lines and circles to 
the curve M has two connected components 5; and Lz. They are simply- 
connected, and all circles from a given component lie on one side of M. 


Definition. A support circle to a curve M is called internal (external) if 
it lies inside (outside) M. 


A vertex of a curve is said to be internal (external) if the curvature 
circle at this vertex is an internal (external) support circle to the curve. 


Corollary 3. Suppose that a simple closed embedded generic curve M ina 
plane has Cy internal and C2 external vertices, T, internal and T2 external 
support circles tangent to M at three points. Then 


(5) CT =2, C, —T, =2. 


This statement follows from Corollary 2. Indeed, if ©,(%2) contains 
internal (external) support circles to the curve M then C; = Se), IB = 
xi(1,1,1), 7 = 1,2. Now, the relations (5) follow from the equalities (4) 
and x(Z;) = il 
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5. ON SUPPORT A;s-SPHERES 


A smooth submanifold in a 4-space can have support As-spheres. The 
points of tangency of these support hyperspheres with the manifold are 
called its global vertices. The above results allow to obtain some equali- 
ties connecting the number of global vertices of a manifold with other its 
differential-geometric characteristics. 

Let M be a smooth closed connected generic submanifold in R4. Then 
the relation (3) implies 
(6) 
2x(5) —2x(1, P15 1) el, 3) -X(1, 1,1, 1) + 2x(3) = x(1; 1) == 3x(M), 


where x(/1,...,/m) is the Euler number of the manifold of support 
A,,---A,,,-planes and spheres to M. On the other hand, we have two 
equalities 


x(1, 3) = x(5) + x(1, 1,3), 
aq Gly| Th A) apa Gl il) tea, 1 


which follow from the incidence relations of the graph consisting of 0- 
dimensional and 1-dimensional strata of the Maxwell set of a generic 4- 
parameter family of functions (see [A]). Therefore, (6) can be rewritten in 
the form 


(8) ~ 5x(11,3) + 5x(1, 1, 1,1,1) = 6 —2x(8) + x(1,1) — 8x(M), 


This relation implies the interesting invariant of strongly conver curves 
in a 4-space (the strong convexity means that every hyperplane intersects 
the curve at most at 4 points counted with the multiplicities). 


Corollary 4. Suppose that a simple closed embedded strongly convex 
generic curve M in a R* has V global vertices, U support hyperspheres 
touching M at three points and having a contact of third order at one of 
them, and P support hyperspheres tangent to M at five points. Then 


2 Oe a 


In particular, 2V + P is greater than or equal to 12. 


Indeed, V = x(5), U = x(I, 13) 1) and Ae) = 
x(1, 1) = x(M) =0 since M is a closed strongly convex generic curve (the 
manifold A(3) and the double covering of the manifold A(1, 1) are the total 
spaces of locally-trivial fibrations over M Ve 


Remark. Using the results of [S1], one can obtain a similar invariant for 
some class of curves in a 5-space. 
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Let us consider a simple closed embedded generic curve 7 in R® having a 
strongly convex projection to a hyperplane. Suppose that -y has V support 
As-planes, U support A;.A1A3-planes, and P support A; A; AA A1-planes. 
Then 2V —U+P = 12 (in particular, 2V +P > 12). The proof is analogous 
to that of Corollary 4. 

The points of tangency of support As-planes with the curve y are its 
flattening points (where the osculating Frenet frame is degenerate). These 
points can be called global vertices of the curve. The above formula implies 
that the number of global vertices of the curve y is equal to V = 6+ (U — 
P)/2. We conjecture that V > 6. It should be noted that the number of all 
flattening points of a smooth closed curve in R®° having a strongly convex 
projection to a hyperplane is greater than or equal to 6 (see [Ar]). 
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On Combinatorics and Topology 


of Pairwise Intersections of 


Schubert Cells in SL,/B 


Boris Shapiro, Michael Shapiro,! Alek Vainshtein 


0. Introduction 


0.1. Topological properties of intersections of pairs and, more generally, of 
k-tuples of Schubert cells belonging to distinct Schubert cell decompositions 
of a flag space are of particular importance in representation theory and 
have been intensively studied during the last 15 years, see e.g. [BB, KL1, 
KL2, Del, GS]. Intersections of certain special arrangements of Schubert 
cells are related directly to the representability problem for matroids, see 
[GS]. Most likely, for a somewhat general class of arrangements of Schubert 
cells their intersections are too complicated to analyze. Even the nonemp- 
tyness problem for such intersections in complex flag varieties is very hard. 
However, in the case of pairs of Schubert cells in the space of complete 
flags one can obtain a special decomposition of such intersections, and of 
the whole space of complete flags, into products of algebraic tori and lin- 
ear subspaces. This decomposition generalizes the standard Schubert cell 
decomposition. The above strata can be also obtained as intersections of 
more than two Schubert cells originating from the initial pair. The decom- 
position considered is used to calculate (algorithmically) natural additive 
topological characteristics of the intersections in question, namely, their Eu- 
ler E?*4-characteristics (see [DK]). Generally speaking, this decomposition 
of the space of complete flags does not stratify all pairwise intersections 
of Schubert cells, i.e. the closure of a stratum is not necessary a union of 
strata of lower dimensions. Still there exists a natural analog of adjacency, 
and its combinatorial description is available, see Theorem D. We discuss 
combinatorics of this special decomposition and some rather simple conse- 
quences for the cohomology and the mixed Hodge structure of intersections 


of Schubert cells in SL,,/B. 


0.2. In order to formulate the main results, let us recall some standard 


1The paper was partially written during the visit of this author to the Department 
of Mathematics, University of Stockholm, supported by the NFR-grant R-RA 01599- 
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notions. Let F,, denote the space of complete flags in C”. Each compete 
flag f can be interpreted both as a Borel subgroup and as a sequence of 
enclosed linear subspaces of all dimensions from 0 to n. The Schubert 
cell decomposition Dy of F, relative to f consists of cells formed by all 
flags having a given set of dimensions of intersections with subspaces of 
f. Thus, we have a family of Schubert cell decompositions parameterized 
by F,. Cells of any such decomposition are in 1-1-correspondence with 
permutations on n elements (see, e.g., [FF]). 


0.3. For any k-tuple of flags f,g,h,... in F,, we introduce the k-tuple 
Schubert decomposition Dj,g,n,... consisting of all nonempty intersections 
of k-tuples of cells one taken from each decomposition Ds, Dj, Dh.... 
Generally speaking, a k-tuple Schubert decomposition is not a stratification 
of the space of complete flags, and its strata can have very complicated 
topology, cp. [GS]. 

Given a family of linear subspaces, their +- completion is defined as the 
set of sums of all possible subfamilies of these subspaces. The +-completion 
of a pair of flags is just the +-completion of the family of all subspaces 
constituting these flags. 

The refined double decomposition RD f,g of the space of complete flags 
relative to a given pair of flags (f,g) is the decomposition into pieces formed 
by all flags with given dimensions of intersections with all subspaces of the 
+-completion of (f,g). The pieces of this decomposition are called refined 
double strata. 


0.4. Remark. The refined double decomposition coincides with some 
special k-tuple decomposition, see §1. 


0.5. Remark. The refined double decomposition RD f.g Subdivides the 
standard decompositions D f, Dg and the double decomposition D fone: 
each Schubert cell with respect to f or g, as well as their pairwise intersec- 
tions, consists of some number of refined double strata. 


0.6. Remark. If g’ belongs to the same Schubert cell of D f as g, then 
RDyj,q is isomorphic to RDy,9, and the isomorphism is induced by a linear 
operator preserving f. 


Theorem A. Each refined double stratum is biholomorphically equivalent 
to the product of a complez torus by a complex linear space. 


0.7. Given a system of coordinates in C”, a flag is called coordinate if 
all its subspaces are spanned by coordinate vectors. Each coordinate flag 
is identified naturally with the corresponding permutation of coordinates. 
The standard coordinate flag is the one which is identified with the unit 
permutation, i.e. for each 7 its i-dimensional subspace is spanned by the 
first 7 coordinate vectors. 

For any two flags f and g in F,, one can always choose a standartizing 
system of coordinates such that f becomes the standard coordinate flag 
and g becomes a coordinate flag identified with some permutation ao. We 
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then say that the pair (f,g) is in relative position 0. Observe that the 
permutation corresponding to the cell containing g in the Schubert cell 


decomposition Dy is exactly a7}. 


0.8. In what follows we need some additional notation related to permuta- 
tions. We write an arbitrary permutation 7 in the form 7 = 7, ...%,, which 
means that m(1)\'=-41,..., 7 (2) = tn. 

A decreasing subsequence in 7m is a subsequence s = (i;,,%j,---523,) 
Sicmhiat amy iaio <j, Se and 2; 2) te 

The reduced length of a decreasing subsequence is equal to the number 
of its elements minus one. The domination of a decreasing subsequence is 
equal to the number of elements i; € a for which there exists an element 
a, € s such that 7 </ and 7; <7. 

The cyclic shift of wrt a decreasing subsequence $ = (2;,,2j2,---523,) 
is the transformation sending 7;, onto 2;,, 2, onto 2;,,..., 17, Onto 2;,_, 
and preserving the rest of the elements. (If s consists of just one element, 
then the transformation is identical.) 
Example. Consider 7 = 6723451 and its decreasing subsequence (7, 3, 1). 
The reduced length is 2 and the domination is also 2, namely, element 6 is 
dominated by 7 belonging to the decreasing subsequence, and element 2 is 
dominated by 3. The cyclic shift of 7 wrt (7,3, 1) takes 7 to 6321457. 


0.9. Till the end of this section we assume that the pair of flags (f, g) is in 
relative position o. Let us enumerate (algorithmically) all strata of RD}, 
using the permutation o. To do this, we apply to o the following n-step 
procedure. 


Main algorithm. 

Step 1. Find all decreasing subsequences in a. Apply to o cyclic shifts 
wrt each of these decreasing subsequences and obtain the set of resulting 
permutations. In each of these permutations block the largest element of 
the corresponding decreasing subsequence. (To block just means that this 
element is ignored on all subsequent steps of the algorithm.) 

Step i. To each permutation obtained on Step 7 — 1 apply the same 

procedure that was applied to 0 on Step 1. Namely, find all its decreasing 
subsequences (disregarding all blocked elements). Make cyclic shifts wrt 
each of these decreasing subsequences, and finally in each of the obtained 
permutations block the largest element of the corresponding decreasing 
subsequence. 
0.10. Remark. The algorithm stops exactly after n steps, since in each 
permutation obtained after 7 steps we get exactly 7 elements blocked (i.e., on 
Step +1 we work actually with permutations on n—i elements). Moreover, 
each permutation with at least one nonblocked element contains at least 
one decreasing subsequence (possibly consisting of just one element). 

The whole procedure for o = 321 is illustrated in Fig. 1. The under- 
lined numbers are blocked and the sequences of numbers on edges present 
the decreasing subsequences chosen. 
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321 


3=C 31-9 322 321=Cx C” 
321 123 


1=C?/\\21=C” 1=C? 


321 if if i 7 y | if if , ii ii if if ii 7 il , 
| Cc eCeOc ec eer oO oc acee ¢ cy ey cy cy 
Figure 1. Illustration of the main algorithm for the case o = 321 


0.11. A chain of permutations is a sequence of n+-1 permutations starting 
with o and such that each consequent permutation is obtained from the 
preceding one as the result of a cyclic shift wrt a decreasing subsequence 
on the corresponding step of the above procedure. 

So, Fig.1 contains 20 chains, which are just paths in the presented tree 
starting from the top element o and going down to the bottom. 

Let us assign to each chain two numbers, namely, its total length equal 
to the sum of the reduced lengths for all the permutations involved (ageu- 
ally, for the corresponding decreasing subsequences of these permutations), 
and its total domination equal to the sum of all dominations. (When we 
calculate the domination for a permutation with blocked elements we just 
disregard them completely.) 


Theorem B. Let (f,g) be a pair of flags in relative position o. Then 
the strata of RDs,9 are in 1-1-correspondence with the chains in the above 
procedure starting with o. The stratum corresponding to a given chain is 
isomorphic to (C*)! x C4, where | is the total length of this chain and d is 
its total domination. 


Example. The structure of each refined double stratum is given in the 
bottom line of Fig.1. 


0.12. For any two permutations a and B denote by Ci, the Schubert 
cell consisting of all flags which are in relative position @ with respect to 
f and by C,,g the Schubert cell of all flags in relative position 8 with 
respect to g. (Warning: in the standard notation Cr — Bae Bend 
Co,8 = 0~'BoB" - B, see 0.7, where B is the subgroup of upper triangular 
matrices.) 

By definition, the cell Ci,a belongs to the decomposition Dys, the cell 
Cz,g belongs to the decomposition D,, and their intersection belongs to the 
double decomposition D he 
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Since the refined double decomposition RD;,, subdivides the decom- 
positions D;, Dg and Dy_q, it is useful to describe all refined double strata 
included in the Schubert cells C),., Co,g and their intersection Cy ,4MCo,g. 

Let us assign to a chain of permutations the following two new per- 

mutations. The first blocking permutation of a chain is the sequence of 
the successively blocked elements, i.e., its 2th entry is the element blocked 
on the ith step of the procedure. The second blocking permutation is the 
sequence of positions on which the successively blocked elements stand, i.e., 
its 7th element is the number of the position where the ith blocked element 
stands. 
Example, see Fig.l. For the chain 321-123 —123 — 123, the first 
blocking permutation is 321 and the second blocking permutation is 321. 
For the chain 321-312 —132 — 132, the first blocking permutation is 231 
and the second blocking permutation is 321. For the chain 321— 321— 
312 — 312, the first blocking permutation is 321 and the second blocking 
permutation is 132. 


Theorem C. A stratum of RD;,, belongs to the Schubert cell Ci,. if and 
only if the first blocking permutation of its chain coincides with a; a stratum 
of RD #9 belongs to the Schubert cell Co,g if and only if the second blocking 
permutation of its chain coincides with B. Therefore, a stratum belongs to 
the intersection C\,4Coz,g if and only if its first blocking permutation is 
a and its second blocking permutation is 3. 


0.13. This theorem leads us to the following modifications of the described 
algorithm producing the refined double decompositions of the Schubert cells 
Ci, Co,g and of their intersection. 


Modification 1. In order to obtain the decomposition of C),,., one must 
consider on Step i, i = 1,...,, only decreasing subsequences starting at 
the ith element of the permutation a. 


Modification 2. In order to obtain the decomposition of Cz,g, one must 
consider on Step i, i = 1,...,, only decreasing subsequences ending at the 
position whose number is equal to the ith element in the permutation (. 


Modification 3. Finally, in order to obtain the decomposition il Oren a) 
Co,g, one must consider on Step 7, i = 1,...,n, only decreasing subse- 
quences starting at the ith element of permutation a and ending at the 
position whose number is equal to the ith element in the permutation (. 


The corresponding three examples are presented on Figs. 2,3. 
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321 
321 123 231 213 
2=c?/ \21<¢ 2=C a=c) / \2zi=c* 2=c* / \21=c 
321 312 | | | | | 
321 312 | i i 213 123 


o Cc cc’ GC pr Cir jr 


321 321 
1=c® 21-CY 31=C° 3212C°xC" 31=C" 321=C*xC* 


321 312 123 213 123 213 
2=0°/ \32=-0° 120% \31=c 2c 1=€" / \21=-¢ 2a 21=C 
| | | [ | | 123 [ [ 
if | | 132 i 213 123 | i 
fo SC CF Ce cxc cxc (cy C’xc (Ge 


Figure 3. Modified algorithm for the cases o = B= 221 
ENN Goo (6) 8)9))| 


A similar process was proposed by Francesco Brenti [Br1] in the case 
Oo = Wp. 
0.14. The following remark is valid for all versions of the main algorithm, 
i.e. for the refined double decomposition of the whole space of complete 
flags, of some particular Schubert cell, or of a pairwise intersection of Schu- 
bert cells. 


Remark. Each step of the above algorithm is interpreted geometrically 
as the projection of flags in the considered set onto a linear subspace of 
the corresponding codimension, see §1. This means, in particular, that 
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restricted chains, i.e. those starting at a permutation with blocked elements 
obtained after Step i, represent the decomposition of F,_; (a Schubert cell 
in F,—~:, or a pairwise intersection of Schubert cells in F,_;, depending on 
the modification of the algorithm). 


0.15. To be more precise, we introduce the following operation on permu- 
tations with blocked elements. Let us consider a permutation on n elements 
with i blocked entries. The reduction of the blocked part from a given per- 
mutation is the operation which forms a new permutation on n—i elements 
in the following way: we exclude all blocked elements and subtract from 
each nonblocked element the number of all blocked elements which are less 
than it. 

Example. The reduction of the blocked part from the permutation 
7563241 gives 4312. 


0.16. Proposition. i) The set of all restricted chains, i.e. chains starting 
at some permutation G obtained after i steps of the algorithm (and thus 
containing i blocked elements), geometrically presents: 


(1) for the main algorithm, the refined double decomposition OF foe: 
relative to the pair in relative position G’, where o’ is obtained from 
& by the reduction of all blocked elements; 

(2) for the 1st modification, the refined double decomposition of the 
Schubert cell Cy,q in Fy—i, where a’ is obtained by the reduction of 
the first i elements from a; 

(3) for the 2nd modification, the refined double decomposition of the 
Schubert cell C3: in Fn—i, where a’ is the same as above and (3’ 
is obtained by the reduction of the first i elements from B; 

(4) for the 3rd modification, the refined double decomposition of Ch,a/M 
Ce" p'- 

ii) Moreover, the geometrical meaning of Step i for any modification of 
the algorithm is the decomposition of the initial object (Fn, Cie, Coe 
or Cia Co,g) into a disjoint union of products of similar objects in 
F,,-;, enumerated by the set of all permutations & obtained after Step 1, 
by (oP Ke) x C4), Here I(G) is equal to the sum of reduced lengths for all 
permutations in the chain starting at o and ending at a, and d(G) is the 
sum. of all dominations in this chain. 


Example. The set of all reduced chains passing through any permutation 
obtained after the first step except for 123 in Fig.1 presents the refined 
decomposition of the space of complete flags Fy = CP! relative to o = 21 
into three strata, namely, two points and C*. The chains passing through 
123 (or o = 12) present the standard Schubert cell decomposition of 2 
into a point and C. 


0.17. Remark. The above results hold in complete generality for the 
spaces of complete flags over any algebraically closed field, or R. In this 
case C* must be substituted by the multiplicative and C by the additive 
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group of the field, cp. [Cu3]. 


0.18. Our next result gives sufficient conditions for the topological “ad- 
jacency” of strata in Ci,¢9Co>,g in terms of combinatorial adjacency, i.e. 
enumerates strata that can have nonempty intersection with the closure of 
some given stratum in C),.4M Cz,8- Since the refined double decomposition 
in general is not a stratification, one has to use this modified notion of 
adjacency. 

Consider two chains of permutations. One of them is said to be less 
or equal than the other one if each permutation of the former is less or 
equal in the Bruhat order than the corresponding permutation of the latter 
(for the notion of the Bruhat order see e.g. [Hu]). The above partial order 
on the set of all refined double strata (or their chains of permutations) in 
Ci,aCz,g is called the adjacency partial order, or the generalized Bruhat 
order. 

By Theorem C, each nonempty refined double stratum of RDyj.9 is 
contained in a single pairwise intersection Cia Cz,g, namely, in the one 
for which a equals to the first and # to the second blocking permutation of 
the corresponding chain of permutations. 


0.19. Theorem D. The closure of a given refined double stratum in 
the corresponding pairwise intersection of Schubert cells belongs to (but in 
general does not coincide with) the union of all refined double strata included 
in the same pairwise intersection such that their chains of permutations are 
less or equal in the adjacency partial order than the chain of the stratum 
considered. 


This theorem enables us to construct rather simple examples of pairwise 
intersections whose refined double decompositions fail to be stratifications. 
In particular, the refined double decomposition of C1234,4931 C4231,4231 
consists of three strata, namely, C* x (C)? and two copies of (C*)> x C, see 
Fig. 9 below. In notations of Fig.9 the closure of the stratum C= (GPS ac 
is nonempty in the stratum B = (C*)? xc. moreover, the closure of C = 
(C*)* x C does not contain the whole stratum A = C* x (C)*; see §7 for 
more examples of this kind. 


0.20. Let us consider the closure pattern of a given refined double stratum, 
i.e. the set of all strata that have nonempty intersection with the closure 
of the given one. (The question about the combinatorial description of the 
closure pattern was raised for a similar situation in [Cu3].) Theorem D 
gives us a necessary combinatorial condition for a stratum to belong to the 
closure pattern of the other one. In 87 we present an example showing that 
this necessary condition is not sufficient. The relation between necessary 
and sufficient conditions motivates the following definition. 

We call a pairwise intersection of Schubert cells CiaNCo,g nice if the 
refined double decomposition RD fg Sives its stratification, and almost nice 
if for any pair St; ~ Sty of refined double strata one has dim St, < dim Sto. 
The rest of CiaMCoz,g are called hard. An example of a hard Cia (NCe.g 
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is given in §7. 


0.21. Let V denote an arbitrary complex quasiprojective variety. Let us 
denote by h?? the Hodge numbers for the usual mixed Hodge structure in 
Hz(V;C), see e.g. [D11, D12, D13). 


Let us define generalized Euler characteristics depending on p and gq: 
a 
k 


and form their generating function called the E?:4-polynomial, or just the 
E-polynomial of V: 


Ey(u,v) = sz xtaeor 
Pd 


Let us denote by E% the E-polynomial of Ci, 9 Cz,g, and by 
CH(a, 8,c) the set of chains of permutations corresponding to the strata 
contained in C1,4Coz,g (see Theorem B). 


0.22. Corollary (of Theorems B and D). i) The Hodge numbers hi? of 
any intersection Ci,Cz,g can be positive only if p = q. 
ii) 
Bee = SD eA (g — 1H, 
chE€CH(a,f,c) 


where z = uv, d(ch) and I(ch) are the total domination and the total length 
of a chain ch, respectively. 


0.23. The same expression ii) can be rewritten as an inductive formula 
using the above remark on the geometrical meaning of our algorithm (see 
0.14). More precisely, let S denote the set of all decreasing subsequences 
in o starting at element a(1) and ending at the position with number B(1), 
(ie., the set of subsequences used on the first step of the construction of 
the refined double decomposition for Ci,4MCz,g). For any decreasing sub- 
sequence s € S, let I(s) and d(s) denote its reduced length and domination, 
respectively. Finally, let a’ and (’ denote the results of the reduction of 
the first elements a(1) and G(1) from @ and f, respectively, o, denote the 
result of the cyclic shift of o wrt s and the reduction of the first element of 
s (see the description of the algorithm and its modifications above). 


Corollary (of Theorems B and D). 
jae = DD 7s) (z x 19) Be" 8 
ses 
Example. 
83214821 _( — eo mee 1)2 822,821 


4321 132 
+ (IPERS + (BRE. 
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0.24. Corollaries 0.22 and 0.23 follow directly from the refined double 
decomposition of Cy,4Cz,g. By results of [Cu3], their natural analogs are 
also valid for any G/B, where G is a semisimple group and B is its Borel 
subgroup. Part i) of 0.22 holds as well for all quasiprojective varieties that 
can be decomposed into quasiprojective pieces satisfying h?’? = 0 if p £ q. 
0.25. The adjacency partial order (see 0.18) allows us to consider different 
filtrations of C},4MCo,g by closed subsets consisting of refined double strata. 
Moreover, one gets the standard filtration of such a kind as follows. 

Let P be a finite poset. We define the height of an element a € P as the 
maximum length of a chain having a as the maximal element. The standard 
filtration of P is its filtration by the subsets P; = {a € P | h(a) < i}, called 
standard subposets. 

Consider now the refined double decomposition of C1aMCo,g. Refined 
double strata are enumerated by chains of permutations, which form a finite 
poset with respect to the adjacency partial order. Theorem D shows that 
the standard filtration of this poset is a filtration by closed subsets. 

Any filtration of Cy,4M Co,@ by closed subsets leads to its Leray spec- 
tral sequence converging to the cohomology of Cia Co,g with compact 
supports. In our case we are primarily interested in filtrations by closed 
quasiprojective subvarieties. Along with the above standard filtration, it 
is often convenient to consider “filtration” by dimension: its ith term is 
the union of all strata of dimension at most 7. (Warning: for an arbitrary 
C1, Cog this filtration is apparently not a filtration by closed subsets.) 
In the case of an (almost) nice Cia Co, the filtration by dimension is 
a filtration by closed quasiprojective subvarieties. Moreover, the following 
statement is valid. 


Theorem E. The Leray spectral sequence associated with the filtration by 
dimension of the refined double decomposition of any (almost) nice pairwise 
intersection degenerates at the second page. 


0.26. Remark. A combinatorial description of d, is unavailable at the 
present moment and apparently is rather complicated. 


0.27. As an application of the E-polynomials we prove the following com- 
binatorial result. 


Proposition. If at least one of the permutations a, or o is the longest 
element wo, then there are no gaps in (complex) dimensions of strata in 
C1,aNCog, t.e. there exist refined double strata of all intermediate (com- 
plex) dimensions between the minimal and the marimal ones. 


0.28. A similar family of decompositions was introduced by V.V.Deodhar 
in the case of intersections Cia Cwo,8, Where wo denotes the longest 
element in an arbitrary finite Coxeter system, see [Del, De2], and was ex- 
tended to all intersections C1,¢9Co,¢ by Ch.Curtis in [Cu3]. These decom- 
positions depend on a reduced expression of the element @ as a product of 
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simple reflections, and different choices of such expressions lead to different 
decompositions. The combinatorial data that codes strata in the approach 
of Deodhar—Curtis is similar to chains of permutations corresponding to re- 
fined double strata, but more lengthy. We have found the correspondence 
between the strata of these two decompositions and proved that the refined 
double decomposition coincides with one of decompositions suggested by 
Deodhar for some particular choice of reduced expression. 

Let us define for any a € G,, a reduced expression of a! of a special 
form, namely, a7! = t9(1)-1ta(1)-2°++t1@~*, where & belongs to G,_1 
and t; is the simple transposition interchanging 7 and 7+ 1. This enables 
us to define inductively a reduced expression, which we call the standard 
expression. 


Proposition . The refined double decomposition of the Schubert cell 
Cia = Baé~'- B coincides with the decomposition suggested by Deodhar 
if one chooses the standard reduced expression of ame 

0.29. Remark. There exist other natural refinements of double decom- 
positions. These other decompositions of geometrical origin apparently 
coincide with Deodhar’s decompositions corresponding to other choices of 
reduced expressions. 


0.30. The starting point of this study was an attempt to calculate the 
cohomology of pairwise intersections of Schubert cells of the maximal di- 
mension (see [SV]). 

The authors are very grateful to Francesco Brenti for discussions of the 
material of this article. Sincere thanks are due to Prof. T.Springer who 
was first to point out to the authors the necessity to check the property of 
being a stratification and mentioned the paper [Boe]. 


1. Refined double decompositions as k-tuple decompositions 


and the very weak Bruhat order 


1.1. In order to study the properties of refined double decompositions, 
let us fix a pair of flags (f,g) in relative position o and consider its +- 
completion. By the definition (see 0.2), the +-completion consists of all 
sums of pairs of all possible subspaces from the flags f and g, see the 
example in Fig.4. Obviously, all the subspaces in the +-completion of 
(f,g) are coordinate (in the corresponding coordinate system, see 0.7) and 
partially ordered by inclusion. 

By the flag completion of the pair (f, g) we mean the set of all com- 
plete flags such that each subspace of each of these flags belongs to the 
+-completion of (f, 9). 

The number of flags in the flag completion of a pair of flags in relative 


408 B. Shapiro, M. Shapiro, A. Vainshtein 


position o is denoted by k(q); it is just the number of all different paths in 
the partially ordered +-completion going from the top to the bottom. For 
example, there are 10 paths (and therefore flags in the flag completion) in 
the diagram shown in Fig.4. 


Figure 4. The +-completion of the pair of flags in relative position 53241. 
Each set of numbers presents the coordinate subspace spanned 
by the corresponding coordinate vectors. 
Subspaces in each row are lexicographically ordered 


1.2. Proposition (an alternative definition of the refined double decom- 
position). The refined double decomposition RD ys, is the k-tuple decompo- 
sition Dy, 5,,..., where f; runs over the set of all complete flags in the flag 
completion of (f,g), and hence k = k(a) is the number of flags in this flag 
completion. 


1.3. Remark. A special n!-tuple Schubert decomposition into intersec- 
tions of n!-tuples of cells taken from Schubert cell decomposition relative to 
each of coordinate flags was introduced in [GS] and called the decomposition 
into small cells. This decomposition and its projections on Grassmanians 
play an important role in matroid theory, since each small cell on some 
Grassmanian is the space of linear presentations of the corresponding ma- 
troid. The decomposition into small cells subdivides each of our refined 
double decompositions. 


1.4. Let us define the following partial order on G,. By an elementary 
descent in a permutation i, ... tn we call any inversion of two neighboring 
numbers 2), 74, such that 

i) it decreases the inversion length of the permutation, and 

ii) 7241 is less than all the elements Cs eee 

The transitive closure of elementary descents defines a partial order on 
G,, which we call the very weak Bruhat order. It is, obviously, a suborder 
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of the weak Bruhat order. 
The only minimal element in the very weak Bruhat order is the identity, 
while the set of maximal elements consists of all elements having 1 in the 


last position. Figure 5 presents the very weak Bruhat order for the cases 
of G3 and Gy. 


4321 
4312 4231 43421 3241 


31 4132 4213 ale 3214 
~ il Ab ag ee 
1 { — 1423 i-— 
‘I 
Ml. 2134 iD 1324 


123 1234 


Figure 5. The very weak Bruhat order on 63 and G4 


1.5. Proposition. The number k(a) equals the number of permutations 
in the lower interval of o in the very weak Bruhat order. 


Proof. Follows immediately from the definition (see 1.1 and 1.4). i 


1.6. Below we present a simple scheme for finding the number k(o), which 
we call a generalized Pascal triangle. Recall that the standard Pascal tri- 
angle (of size n) can be represented as a box diagram having n rows and 
n columns. Each box is a square. The first row contains n boxes, the 
second row contains n — 1 boxes, and so on; the first column contains 1 
box, the second column contains 2 boxes, and so on. The diagram is filled 
by integers in the following way. We start at the unique box of the first 
column and fill it with 1. Next, the contents of any other box is defined to 
be the sum of the contents of its western and north-western neighbors (if 
any). Proceeding in accordance with this rule, we first fill all the boxes of 
the second column by ones; this gives a possibility to fill the boxes of the 
third column, and so on (see Fig. 6). The elements in the ith column of 


_— 


the triangle are just the binomial coefficients ( BL) 0<j <i-—1, and their 


sum is 2*~}, 
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Figure 6. Pascal triangle for n = 6 


To construct the generalized Pascal triangle of type a, we define L, ye 
1 <1 <n, as the number of elements 1; Ol o such that 7 > and ee ie 
We now start from the diagram of the standard Pascal triangle and remove 
the horizontal partitions between the last (from left to right) Z,(1) boxes 
of the first and the second row, then the horizontal partitions between the 
last L,(2) boxes of the second and the third row, and so on. As a result, 
we get a new box diagram; its boxes are no longer squares, but rectangles 
of width 1 and arbitrary height. It is easy to see that the number of boxes 
in the diagram equals n plus the inversion length of co. 
Example. Let n = 6, o = 426135. Then the vector of L,’s is 
(2,3,0,2,1,0). The diagram of the corresponding generalized Pascal tri- 
angle is presented in Fig.7a. The inversion length of o is 7, and the number 
of boxes in the diagram is 6+ 7 = 13. 


Figure 7. Generalized Pascal triangle of type 426135 


The diagram thus constructed is now filled by integers according to 
the same rule as above. The only difference is that a box can have more 
than one western neighbor; in this case we have to sum the contents of 
all western neighbors, and that of the north-western one. As before, the 
table is filled column by column, starting from 1 placed in the unique box 
of the first column. The result of this procedure for the above example is 
presented in Fig.7b. 


1.7. Proposition. The number k(o) equals the sum of the numbers in 
the last column of the generalized Pascal triangle of type a. 


Proof. Let us define two projections ™,0!: Gp > Gp_y. The first of them 
takes 11... iplipgg...in to 47 —1...%,—-—1ixyo—-1...i, —1; the second one 
takes aig...in to 13...72, where wy = 1; ifi; <a and i =1;-1 Mos Soe, 
It is easy to see that for o =i, .. .tn one has 


nfo) = { Mme) +H) sta 


(*) k(m(c)) = k(m(c)) otherwise © 
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Indeed, let i; #1 (the case 21 = 1 is trivial). By Proposition 1.5, we have 
to count the number of permutations in the lower interval of o in the very 
weak Bruhat order. There are two types of such permutations: those in 
which 7, and 1 form an inversion, and those in which they do not. By the 
definition, any path from o to a permutation of the first type in the very 
weak Bruhat order avoids elementary descents transposing i; (otherwise 
condition ii) of 1.4 would be violated). Thus, 7; remains all the time at 
the first position of any permutation in such a path and does not influence 
any of the descents used. Therefore, the number of the permutations of the 
first type equals k(71(c)). 

On the other hand, any path from o to a permutation of the second 
type contains elementary descents involving i;, and the first such descent 
on any of these paths is the transposition of 7; and 1. Thus, 1 occurs at 
the first position, and remains there all the time. Therefore, the number of 
the permutations of the second type equals k(7(c)). 

Let now b(c) denote the sum of the elements of the last column in the 
generalized Pascal triangle of type 0; we shall prove that b(o) satisfies the 
same equation as k(o). Again we first assume that 1 is not in the first 
position of o. Hence, the second column of the generalized Pascal triangle 
consists of two square boxes, and each of the boxes contains 1. Let us 
consider two other generalized Pascal triangles with the same diagram that 
the initial one; the first of them contains 0 in the first box of the second 
column and 1 in the second box, while the second triangle contains 1 in the 
first of these boxes and 0 in the other one. All the other columns of the both 
triangles ate filled according to the rule 1.6. Since the rule is additive, we 
get that the contents of any box of the initial triangle (except for the box in 
the first column) equals the sum of the contents of the corresponding boxes 
in the two new triangles constructed. However, the first of these triangles 
corresponds bijectively to the generalized Pascal triangle of type 71(c) (it is 
sufficient to shorten by 1 the heights of all uppermost boxes; if such a box is 
of height 1, then its contents is 0, and we just remove it). In a similar way, 
the second of the triangles corresponds bijectively to the generalized Pascal 
triangle of type 7(o) (we again remove certain square boxes containing 0 
and shorten by 1 certain boxes of height > 1). Therefore, b(c) satisfies the 
first line of (*). The case o = 1i2...%n is again trivial, since in this case 
the second column of the corresponding generalized Pascal triangle contains 
only one box. 

Finally, for the trivial permutation 1 € G one has k(1) = 1, while the 
generalized Pascal triangle of type 1 consists of one box containing 1, and 
so b(1) = 1. Therefore b(c) and k(o) satisfy the same equation with the 
same initial values. | 
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2. Quotients of cells and refined double strata 


2.1. Lemma. Let K, L, and M be a triple of linear subspaces in some 
linear space. Then 


(+) dim((K/L) Nn (M/L)) 
= dim((K + M)N L) — dim(K NL) — dim(M nL) + dim(K n M). 


Moreover, if M contains L, the above formula is simplified to 


(+*) dim((K/L) M (M/L)) = dim(K NM) - dim(K NL). 


The proof is obtained by a straightforward usage of inclusion and ex- 
clusion of vectors of an appropriate basis. 


2.2. Given a complete flag f and a subspace L in C”, we denote by fr the 
complete quotient flag, that is, the flag in the quotient space C"/Z obtained 
by taking consequent quotients of all subspaces constituting f (and ignoring 
occasional coincidences). 


2.3. Assume that C is a cell of the Schubert cell decomposition Dy and 
L C C® is a linear subspace. We denote by Cy the set of all flags in C 
containing L as a flag subspace. 


Lemma. If C, # 0, then the set Cr/L of quotients of all flags in Cr 
is isomorphic to some cell in the Schubert cell decomposition Ds, of the 
space of complete flags in C"/L. Moreover, if the flags in C are in relative 
position a to f and the dimension of L is 1, then flags in Cr/L are in 
relative position a’ to fr, where a! is the permutation on n —i elements 
obtained by the reduction of the first i elements of a. 


Proof. The set C,/L belongs to some cell C’ of the decomposition Dy, . 
Indeed, let g be some flag in Cr, let M be its subspace containing L and 
let K be a subspace of f. Then dim((K/L)M(M/L)) is equal to dim(kK 
M) — dim(K NL) by («*), and thus does not depend on the choice of 
g in Cy. Conversely, any flag f’ € C’ is the quotient of some flag fe 
Cr. The corresponding flag f € Cy can be constructed as follows. Its 
subspaces included in L can be taken from an arbitrary flag in Cz, while 
those containing L are the inverse images of the subspaces of the considered 
quotient flag f’. An easy check shows that i belongs to Cz, since it contains 
L and has the necessary dimensions of intersections with the subspaces of 
Ff. In order to prove that a’ is obtained from a by the reduction of its 
first 2 elements, one must consider a standardtizing basis for the (f, g) (see 
0.7) and take the quotient by the 7-dimensional subspace of g. In this case 
everything is obvious. | 
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2.4. Corollary. The assertion of Lemma 2.3 holds for any k-tuple Schu- 
bert decomposition, i.e. if St is a nonempty stratum of Dyson... and Sty is 
its nonempty subset of all flags containing a linear subspace L, then the set 
St, /L of quotients of flags in Sty, is isomorphic to a stratum in the k-tuple 
decomposition Df, gr ,hy,... 


2.5. Remark. The same holds for refined double decompositions, since 
they are particular cases of k-tuple decompositions, see Proposition 1.2. 


2.6. Remark. If L is one-dimensional, then C,/D = Cy. 


2.7. Proposition. Let h1 and h? be two flags lying in the same refined 
double stratum of RD}, and let L} and L? denote the subspaces in h! and 
h2, respectively, of the same dimension. Then there exists an isomorphism 
®: C"/L1 — C"/L? of the quotient spaces that sends the quotient flag fr 
onto fr2 and the quotient flag gz: onto gr2. Moreover, the quotient flags 
(ht) and h2, belong to the same refined double stratum of the refined 
double decomposition RDs,,,9,2- 


Proof. Such an isomorphism © exists if the pair of quotient flags (fz, 9 1) 
is in the same relative position as the pair (fr2, 9z2). Thus, we must check 
that for any two subspaces K and M of f and g, respectively, dim((K/L')n 
(M/L)) = dim((K/L*)N(M/L?)). Indeed, if we substitute L? for L in (+), 
then we get the same four terms in the right hand side as if we substitute 
L? (this follows immediately from the definition of refined double strata). 
To prove that such an isomorphism ® takes the quotient flag hi, to 
the same refined double stratum of RD;,,,9,. where h2, lies, we must 
show the following. Given any subspace K in the +-completion of the 
pair (f,g) and subspaces I} > L and L? > L? of the same dimension 
that belong to h! and h?, respectively, prove that dim((K TGV a i 
dim((K/ L?)n(L?/L?). Using (+*), we rewrite the above relation as dim(KN 
L})—dim(KOL) = dim(KNL?) —dim(K OL”), which follows immediately 
from the definition of the refined double decomposition. B 


3. Stabilizer of a pair of flags 


and its action on the projective space 


3.1. Let us consider the subgroup Gz = Bno!Bo, which is the stabilizer 
of both flags f and g (recall that B denotes the subgroup of upper triangular 
matrices). The subgroup G,, is given by the following relations: the matrix 
entry a;;, 7 < Jj, vanishes if and only if the numbers 7 and j form an 
inversion in @, i.e. i occurs in o further than 7. 
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3.2. The subgroup G, acts on strata of all four decompositions D;, Dg, 
D;,g and RDys,4, since it preserves both f and g. We first consider the 
natural action of Gz on the projective space P”™—! of all lines in C”. 


Lemma. The set of all G,-orbits on P"-} corresponds bijectively to the 
set S of all decreasing subsequences ino. The orbit Oso corresponding to a 
decreasing subsequence s is biholomorphically equivalent to (C*)Hs) x C4Us) | 
where I(s) is the reduced length of s and d(s) is its domination. Moreover, 
for each orbit Os,o there exists a subgroup Gs.¢ © G, biholomorphically 
equivalent to (C*)'(9)+1 x C4(s) such that its quotient by the scalar matrices 
(isomorphic to O,,.,) acts on Os,o freely and transitively. 


Proof. Let s = (tip eeentia a) be a decreasing subsequence in o and 
D = (j1,32,---; ja(s)) be the set of elements in o dominated by elements 
of s. Then the orbit O,., consists of all lines whose spanning vectors in 
an appropriate standarizing basis for (f, g) have arbitrary nonvanishing 
coordinates with the numbers 4j,... »%1(s)41 and arbitrary (possibly vanish- 
ing) coordinates with the numbers Jis+++,Ja(s); the rest of the coordinates 
vanish identically. Obviously, Os,¢ is an orbit of G, and has the form 
(Ge C4(s), see 3.1. The orbit corresponding to the decreasing sub- 
sequence s contains the distinguished line p, whose spanning vector has 
1’s for all 1; € s and 0’s for the rest of coordinates. In order to describe 
Gs,o © Gz we consider for each element i, from s the (possibly empty) 
subset J;, of elements in D dominated precisely by 7,, i.e. all the elements 
that are dominated by i, but not dominated by 71,...,%,-1. Consider now 
a subset in Gy consisting of the matrices with arbitrary entries in positions 
(tx, Jz), arbitrary nonvanishing entries in positions (i, ix), 1’s on the rest 
of the main diagonal and zeros elsewhere. This set of matrices forms a sub- 
group Gs, in Gg, which topologically is (C*)(9)+1 x C45). The subgroup 
Go acts freely and transitively on the set of all spanning vectors of lines 
in Os,¢; one can check it by multiplying elements of Gs,o by a spanning 
vector of the distinguished line in Os,¢- Thus, the quotient of G s,o by the 
scalar matrices acts freely and transitively on O, ,. 

A straightforward check shows that for any line (which corresponds to 
a point on P”~!) there exists an appropriate decreasing subsequence s in 
a such that the orbit Os,¢ contains this line. a 


3.3. Remark. All lines belonging to an orbit Os,¢ have the same dimen- 
sions of intersections with any subspace of the +-completion of (f,g). And 
conversely, lines with a given set of dimensions of intersections with the 
subspaces of the +-completion belong to the same orbit. 


3.4. Lemma. i) For any line L in an orbit Os,o the quotient flags fr and 
9x are in the same relative position o, defined by the cyclic shift of o wrt 
s and the reduction of the first element of s. 
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ii) The set F® of all flags in F,, containing lines of Os, is diffeomorphic 
to OF x Fi 


Proof. i) The independence of the relative position of fr, and gy, of the 
choice of L in O,,, follows from the transitivity of the action of Gs¢ On 
Osc. The relative position of f, and gz can be calculated easily if one 
takes L equal to the distinguished line p, € Os,¢ (see the proof of Lemma 
Ze 

ii) The set of all flags containing a fixed line L is isomorphic obviously to 
the set of all flags in C"/L, i.e. to F,_-1. Let us denote by v: F, > pee 
the natural map sending each flag onto its 1-dimensional subspace. The 
restriction of the projection v onto the set F* defines the structure of a 
fiber bundle with the fiber F,,_1 and the base O,.. This bundle is trivial 
for the following reason. The transitive and free action of G,,. modulo 
scalar matrices on F'* defines another structure of a fiber bundle on Fs, 
which is transversal to the first one. Namely, its fibers are diffeomorphic 
to O;,¢ and its base can be chosen as the set of all flags containing the 
distinguished line in O.,¢-. | 


4. Proofs of Theorems A and B 


4.1. Proof of Theorem A. We proceed by induction on n. The space 
F) coincides with CP; any pair of flags is just a pair of two (possibly 
coinciding) points. Each refined double stratum is a point, C*, or C (if the 
points coincide). 

Let us consider now some refined double stratum St consisting of all 
flags with a fixed set of dimensions of intersections with all the subspaces 
of the +-completion of (f,g). We focus first on conditions satisfied by the 
1-dimensional subspace of any flag in St. This j-dimensional subspace must 
belong to some coordinate subspace and avoid in it at least one (but possibly 
more) coordinate hyperplanes. The set of all such lines is diffeomorphic to 
the product of a complex torus by a linear space. Actually, each of these 
lines belong to the image (St), i-e. is the j-dimensional subspace of some 
flag in St. Moreover, the restricted projection v: St > v(St) is a trivial 
fiber bundle, see Lemma 3.4. By Remarks 2.5-2.6, the set all flags in St 
with a fixed line is isomorphic to a refined double stratum in the space F,,-1 
of all complete flags in C"-!. Thus, St is diffeomorphic to the product of 
a complex torus by a linear space by some refined double stratum in F;,-1. 
Since any refined double stratum in F,,_1 has the necessary form by the 
inductive hypothesis, Theorem A is proved. | 


4.2. Proof of Theorem B. Let us consider the space of complete flags 
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F,, and a fixed standard pair (f,g) in relative position o. We present an 
inductive construction of the refined double decomposition RD fig Of the 
space of complete flags, which exactly fits the main algorithm. The space 
F,, is decomposed into the disjoint union of F*, where F® is the union of 
all flags whose 1-dimensional subspaces belong to the orbit Oo vblius. 
the set of all F'*’s is in 1-1-correspondence with the set of all decreasing 
subsequences in o, and hence with the branches of Step 1 of the algorithm. 
Each F* is diffeomorphic to Osc X Fn—1, by Lemma 3.4. Let us now 
subdivide each F'*. Namely, take a fiber (which is isomorphic to 1B i))) be 
F* over some line L € O,, and take the quotient flags fr and gz in this 
fiber. We consider their stabilizer G,, and its orbits on the space P"-2, 
Choosing an G,,-orbit on P™~? corresponding to a decreasing subsequence 
s’ of os, we fix the dimensions of intersections of the 1-dimensional subspace 
in a variable quotient flag with the +-completion of (ft,9L). Subsets of 
fibers that are mapped to this orbit form a fibration over Os,¢. Moreover, 
the set F.- of flags in all fibers that are projected on this orbit forms a 
fibration over Oso X Os'c,- Since Gz modulo scalar matrices acts freely 
on O,,., see Lemma 3.2, this fibration is trivial. Fixing the orbits of G, 
and Gz, is equivalent to fixing the dimensions of intersections of 1- and 
2-dimensional subspaces of a variable flag with the +-completion of (f,9). 
Arguing inductively along the same lines, we get a sequence of fibrations 
F*:5"+-- over the products of tori and linear subspaces. Each of these sets is 
characterized by the following conditions: the dimensions of intersections 
of the consecutive subspaces (from 0 up to some dimension) of a variable 
flag belonging to such a subset with the +-completion of (f,g) are fixed. 
Therefore, after n steps one sees that the sets FS’ are precisely the 
refined double strata diffeomorphic to products of tori and linear subspaces, 
and they are in 1-1-correspondence with chains of permutations in the main 
algorithm. 5 


4.3. Remark. The main algorithm describes this sequence of operations 
using the description of G,-orbits in Lemmas 3.2, 3.4. To avoid the reduc- 
tion of elements and to keep track of elements in the original permutation 
we prefer to use blocking instead of the reduction. 


5. The image of a Schubert cell in the projective space 


and Theorem C 


The proof of Theorem C and the modifications of the main algorithm 
follows easily from Theorem B and several additional statements. 


5.1. Lemma. Let Cia be a Schubert cell in Dy and let a= see 
Then the image v(C,q) is the (k —1)-dimensional affine subspace in P?-1 
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consisting of all lines that belong to the k-dimensional subspace of f but do 
not belong to its (k — 1)-dimensional subspace. 


Proof. Follows immediately from the definition of C1,q. |_| 


5.2. Remark. Quite similarly, ifC,,g € D, and G(1) = m, then the image 
v(Cz,g) C P"~} is the (m—1)-dimensional affine subspace of all lines lying 
in the m-dimensional subspace of g, but not in its (m — 1)-dimensional 
subspace. 


5.3. Lemma. Under the assumptions of Lemma 5.1 the set Sg of all 
G,-orbits constituting v(Ci,«) corresponds to the set of all decreasing sub- 
sequences in a having the following two additional properties: 

i) any subsequence s € Sg contains the number k; 

ii) any s € S, does not contain any number greater than k. 


Proof. By Lemma 5.1, we must choose only those orbits that contain lines 
lying in the k-dimensional subspace of f, but not in its (k — 1)-dimensional 
subspace. This is guaranteed exactly by conditions a) and b) above. | 


5.4. Remark. Similarly, let o =71...in and B(1) = m, then the set Se 
of all G,-orbits constituting v(C,,g) corresponds to the set of all decreasing 
subsequences in o having the following two additional properties: 

i) any s € S® contains the number 7); 

ii) any s € S® does not contain the numbers tm41,-.-,%n- 


5.5. Corollary. The set S8 of all G,-orbits constituting v(C1,a)NUV(Co,2) 
corresponds to the set of all decreasing subsequences in o located on the 
interval between the numbers k andi and containing both of them. 


So, if k = im, then there is only one decreasing subsequence consisting 
of the element k = im. If k > im, then the set S® is empty. The topology 
and the dimensions of these orbits are described in Lemma 3.2. 


5.6. Proof of Theorem C. By definition, the refined double decompo- 
sition RD; of the space F, subdivides each Schubert cell Cia, Co,g, and 
their intersection Cj,4NCo,g- Statements 5.1-5.3 describe explicitly the de- 
creasing subsequences that must be used on a step of the algorithm in order 
to select those refined double strata which are contained in one of the above 
three types of spaces. We also know what happens to the permutations a 
and f after a single step of the algorithm. The modifications given in In- 
troduction just present this sequence of operations formally, using blocking 
instead of the reduction. | 


5.7. Proof of Proposition 0.16. Part i) of the proposition is just an 
obvious corollary of Theorem B and the description of the modifications of 
the algorithm. Indeed, all restricted chains starting at some permutation 


418 B. Shapiro, M. Shapiro, A. Vainshtein 


@ are obtained by the same procedure as the one for o itself ignoring all 
the blocked elements in & and (in modifications of the main algorithm) the 
first 7 elements in a and £. Part ii) is obtained by a slight generalization 
of Lemma 3.4 and the proof of Theorem B via an inductive argument. 


6. Adjacency of orbits in P”-! and Theorem D 


6.1. Let us describe combinatorially the adjacency of G,-orbits on P"—!, 
We consider the following partial order on the set of decreasing subse- 
quences in a. A decreasing subsequence sj is said to be less or equal than a 
decreasing subsequence s2 (notation s; + 82) if for each element of s, there 
exists an element in s2 that either dominates it or coincides with it. 


Figure 8. The natural partial order on the set 
of decreasing subsequences in 83427561 


The restrictions of this partial order to the sets Sa, S?, and S8 define the 
corresponding partial orders on these subsets. 

Example. The partial order on the set of the decreasing subsequences 
starting at 8 and ending at the last position in the permutation ¢o = 
83427561 is shown in Fig. 8. 

The mazimal decreasing subsequence of o, denoted by smax(a), is ob- 
tained in the following way. Its first element is the maximal element of oc. 
The ith element of smax(o) is the maximal element of o standing to the 
right of the (i — 1)th element of Smax(@). Similarly one defines the maximal 
decreasing subsequence of any subset of elements in a. 


6.2. Lemma. The partial order + describes the adjacency of the G,- 
orbits on P™1, v(Cy,a), u(Co,g), and U(Ci,a)Mv(Co,g), respectively. In 
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each of these cases there is no gaps in dimensions, i.e. there exist strata of 
all complex dimensions between the minimal and the mazimal. 


Proof. The statement concerning the adjacency follows immediately from 
the description of orbits, see Lemma 3.2. The absence of gaps in dimen- 
sions is settled by the following observation. It suffices to consider the first 
case; the other cases are similar. Take any decreasing subsequence s corre- 
sponding to an orbit of a dimension bigger than one. In this case s contains 
either more than one element, or a single element having a nonempty set 
of dominated elements. We now construct a decreasing subsequence s’ cor- 
responding to an orbit whose dimension is less by one, and which lies in 
the closure of the first orbit. If s consists of just one element, then s’ is 
the maximal decreasing subsequence of the set of dominated elements, see 
the definition in 6.1. If the number of elements in s is bigger than one, 
we replace the last element by the maximal decreasing subsequence of the 
set of elements dominated precisely by the last element, but not by the 
previous ones. If this set is empty then we just drop the last element. All 
checks are straightforward. fi 


6.3. Consider a 1-parameter family of subspaces L(t), ¢ € [0,1], such 
that for all ¢ € [0,1) the subspaces L(t) have a given set of dimensions of 
intersections with all subspaces in the +-completion of (f,g), and (1) has 
the same set of dimensions of intersections with the individual subspaces of 
f and g, but possibly bigger dimensions of intersections with the rest of the 
subspaces in the +-completion. Denote by a(t) the permutation presenting 
the position of 97/1) relative to fia): 


Lemma. For any t € (0,1) one has 7(t) = (1). 
Proof. The Bruhat order > describes the adjacency of Schubert cells. 


Thus, the assertion of the lemma is equivalent to the following inequality: 
for any two subspaces K and M of f and g, respectively, dim((K/L(1)) 9 
(M/L(1))) > dim((K/L(t)) N (M/L(t))). Applying formula (*) of §2 to 
both sides of the inequality and taking into account the conditions dim( KN 
L(1)) = dim(K N L(t)) and dim(M nN L(1)) = dim(M n L(t)), we get ae 


necessary result. 


6.4. Proof of Theorem D. Let us show that the closure of any given 
stratum St € C1aMCo,g is contained in the union of strata whose chains 
are less than the chain of St in the adjacency partial order, see 0.18. Indeed, 
consider a path 7: [0,1] 4 Cia Co, such that 7[0,1) belongs to St and 
(1) belongs to the boundary of St. This means that there exists at least 
one subspace L(1) in y(1) such that the dimension of its intersection with 
some subspace in the +-completion of (f,g) is bigger than that for the 
subspaces L(t) of the same dimension in y(t), t € [0,1). Now, applying 
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Lemma 6.3 we get precisely the necessary statement. | 


7. Combinatorial versus topological adjacency 


7.1. Theorem D gives an obvious sufficient combinatorial condition for the 
refined double decomposition of C1,.a9Co,g to define its stratification (i.e., 
for the closure of each stratum to belong to the union of strata of lower 
dimensions). 

Criterion. If for any two refined double strata St; and St in Cia mOes 
such that Sty ~ St2 the dimension of Sty ts less than that of Sto, then the 
refined double decomposition provides the stratification of CaM Coe 


7.2. Examples. All intersections of top-dimensional Schubert cells in F, 
are stratifications. On the contrary, the simplest example of nonstratifi- 
cation is provided by the refined double decomposition of the intersection 
C1234,4931 O C4231,4231; it consists of the three strata, namely, of C* x C? 
and of the two copies of (C*)? x C. The intersection of the two opposite 
top-dimensional Schubert cells in F's; contains the fragment shown in Fig.9a, 
which apparently shows that it is not a stratification, Still another example 
is shown in Fig.9b. This example shows also that not all branches of the 
algorithm reach the bottom level. 


54321 52341 
51=¢" 531<CxC° s2=(cy 541=(C)C 
531=(C}xC 
14325 34125 21345 32145 42315 
432=(C)C 42x07 3=¢? 31=¢" 21=(C’y 4=C 
13245 32145 21345 12345 21345 42315 
32=C%C 3c a? 4? 4-0 41=C* 214Cy 
12 345 12345 21 345 12 345 21345 12 345 21 345 
2 2=C 2° 2=c° mee 
12345 12345 21345 LI 21345 2 1345 
A B A B iG 


a) b) 


Figure 9. Examples of nonnice refined double decompositions. 
a) a nonnice fragment for a = 8 =o = 54321: 
b) nonnice decomposition for a = 53421, 6 = 53412, o = 5234] 
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Let us describe also in more detail the above simplest example. We 
use projective complete flags on CP? instead of complete flags in the linear 
space C*. The strata are defined by the following conditions: 

A = (C*) x C? corresponding to 4231 — 1234 — 1234 — 1234 — 
1234: the point of the variable flag h belongs to the line containing the 
points of f and g, and the line of h belongs to the 2-plane containing the 
lines of f and g. 

B = (C*)? x C corresponding to 4231 — 2134 — 1234 — 1234 — 
1234: the line o h belongs to the 2-plane containing the lines of f and g, 
and the point of h does not belong to the line containing the points of f 
and g. 

C = (C*)® x C corresponding to 4231 — 3214 — 3214 — 1234 — 
1234: the line of h passes through the origin and the point of h does not 
belong to the 2-plane containing the lines of f and g. 

If we want to get a stratification, we must split B into B; and Bo, where 
B, is given by an additional condition that the line of h passes through the 
origin and Bz is given by the condition that the line of h does not pass 
through the origin. We also split A into A; and Ap given precisely by the 
same additional conditions as B, and Bo, respectively. Then Ai, which is 
diffeomorphic to C* x (C \ {2 points}), coincides with the closure of C in A 
and B, coincides with the closure of C in B. Pay attention to the fact that 
A, is no longer the product of a complex torus by a complex linear space. 


7.3. Using the previous example one can construct the following example, 
which reveals an essential difference between combinatorial and topological 
adjacencies. Namely, we take the refined double decomposition of the in- 
tersection C12345,54231 NM C54321,54231 and consider the two strata A and B 
defined by the following conditions: 

A = (C*)®xC corresponding to 54321 — 42315 — 32145 — 32145 — 
12345 — 12345; 

B = (C*)* x C? corresponding to 54321 — 41325 — 21345 — 
12345 — 12345 — 12345. 

Then B ~ A, but B does not intersect the closure of A, which means 
that A is not adjacent to B. 


7.4. To complete the section, we provide the only known to the au- 
thors example ofa hard Cia N Cop. The intersection C'123456789,987654321 M 
Co87654321,987654321 contains the following two strata: 


987654321 27, 785614329 ~ 765413289 


653412789 —--> 543216789 5431, 431956789 22 
931456789 ~ + 213456789 > 123456789 
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and 


987654321 —", 781654329 585482, 7g15.43289 785482, 


6514327389 “8”, 541326789 422, 431256789 22, 
231456789 2+ 213456789 24, 123456789. 


(The numbers above the arrows present decreasing subsequences used on 
the corresponding steps.) The first chain is bigger in the adjacency partial 
order than the second chain, i.e. its permutations are bigger or equal in 
the usual Bruhat order than the corresponding permutations in the second 
chain. The first stratum has the form (C*)”” x C8, while the second has 
the form (C*)”* x C7. Thus, we get an example of a hard Cia iCe a 
7.5. Remark. Several natural conjectures about the combinatorics of 
refined double decompositions turned out to be wrong. For example, let 
us consider the set S C S used on the first step of the algorithm in the 
decomposition of C1,aCz,g such that for any decreasing subsequence s € & 
the intersection Ci, M Cz,,9' is nonempty. A natural conjecture would be 
that S is an interval in the partial order + on S, see 6.1. This turns to 
be wrong in the examples C12345,546213 M C654321,654213 and Cj9345,53412 M 
C'52341,53412- Moreover, in the latter case the set S consists of 51 = C*, 
531 = (C)*? x C and 541 = (C)*? x C2, and thus even has a gap in 
dimensions of its elements. 

Several combinatorial conjectures based on our consideration and ques- 
tions connected with refined double decompositions are presented in §11. 


8. On the mixed Hodge structure in the cohomology 


of intersections of Schubert cells 


8.1. Remark. All the elements in the pth cohomology with compact 
supports of (C*)* x C! have Hodge indices p — (k + l),p — (k +1), and its 
E-polynomial (see 0.21) equals (z — 1)*z!, where z = wv. 


8.2. The proof of Corollary 0.22 is based on the following lemma. 


Lemma (see e.g. [Du]. Let Y C X be a closed quasiprojective submanifold 
of @ quasiprojective manifold X and let U = X\Y. Then the exact sequence 
of the pair (X,Y) for the cohomology with compact supports 


(t) Se IR(0)) — 18 OO) HEY) S (Uae 


is an exact sequence of Hodge structures, that is, all differentials respect 
Hodge indices. 
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8.3. Proof of Corollary 0.22i. Applying Lemma 8.2, one gets a simple 
inductive proof of the first part of Corollary 0.22. Indeed, let us consider 
the standard filtration F! Cc F? C--- C F™ =Ci4MCz,g, see 0.24, where 
F' is the union of all minimal refined double strata in the adjacency partial 
order, F? is the union of all minimal strata and all strata that are adjacent 
only to strata in F!, i.e. they form two bottom levels in the adjacency 
order, and so on. The difference between any F’ ? and F*—! is the disjoint 
union of strata, by Theorem D. Now we prove (by induction on i) that all 
F* have h2! = 0 if p ¥ q. Indeed, by Remark 8.1, F 1 which is the disjoint 
union of refined double strata, has this property. Next, we consider the 
long exact sequence ({) for the pair (F° 2 F1) and use Lemma 8.2 and the 
fact that F? \ F! is again the disjoint union of strata to get the necessary 
property for F?, and so on. | 


Remark. Exactly the same argument works for any quasiprojective V = 
JV; where V; satisfy hY7?(Vi) = 0 for p # q. 


8.4 Lemma (see [Du]. Jf a quasiprojective V is decomposed in the dis- 
joint union V = U, Vi, where Vi are also quasiprojective, then x”4(V) = 
¥o,x?4(Vi) for all p,q, or equivalently, Ey (u,v) = do; Ev, (u,v). 

8.5. Proof of Corollary 0.22ii. To prove 0.22ii it suffices to apply 
Lemma 8.4 and Remark 8.1 to the refined double decomposition of any 
Cre M CoB: 


8.6. To prove Corollary 0.23 we need the following additional statement. 
Suppose X is an arbitrary complex manifold; denote 


Hy) = dx (X)2". 


Lemma. Let X =U xY and the following assumptions be true: 
1) x4(Y) =0 fort #5; 
2) h2(U) =0 fort # 3. 
Then i) Ix (z) = Tly(z)Ily(z), 
ii) x4 (X) =0 for 1 Aj. 
Proof. Evidently, assumption 2) yields 
nii(X) = So nPEU)nicRd-av) = SOMA? PY). 
Pal pl 
Hence, 
Pteo = ee i= > Gar) GD" ae 
k p,l k 
= SI(-intr Uy PI-P(Y) = DPOF), 
l Pp 


P; 
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Taking into account this relation, we see that assertion ii) follows immedi- 
ately from assumption 1). Next, 


Naa ei i") a) 


= (Le) (N02) = (omv(e), 
and assertion i) follows. Py 


8.7. Proof of Corollary 0.23. To prove the inductive formula of 0.23 
we use part ii) of Proposition 0.16 for one step of the algorithm, and get 


Cia a) Co,p a I] Os,0 x (Cia MN Co,,8’)s 
ses 


where O, , is the orbit in P?—! corresponding to the decreasing subsequence 
sand Ci q ACG,,g’ is the intersection of Schubert cells in F,,_1, see Lemma 
3.2. Then, from Corollary 0.22i, Lemma 8.6, and Remark 8.1 we see that the 
E-polynomial of One x (Cra Cz,,g") is equal to the product of z24() (z- 
nie and Ec, Pee ee Thus, we obtain the following inductive formula 
for the E-polynomials: 


BG = Boy rCag(tv) = Y- 24(2 — 1) go's" 
scS 


8.8. The Leray spectral Sequence can be used successfully for a filtration 
of a topological space X satisfying the following condition. Let F CX bea 
closed subspace of the filtration considered. One has the following natural 
short exact sequence of sheaves: 0 — jC + C > ;,C 5 0, where j and g 


denote the obvious inclusions F 4 X &@ x \F and C is the usual constant 
sheaf on X. The inclusion F <> X is called good if the sheaves ;,C and Ae 
coincide with constant sheaves on F and X \F, respectively. An example of 
a bad inclusion is the inclusion of the line z = 0 in the space R? \ {y 4 0}. 
Inclusions of quasiprojective varieties are always good. Consider now a 
topological space that is an arbitrary union of refined double strata in 
some C1,4MC,,g. Warning: arbitrary unions of refined double strata are not 
always quasiprojective varieties. At the same time, all lower intervals in the 
adjacency partial order, unions of lower intervals and differences between 
such unions present quasiprojective varieties. The adjacency partial order 
defines the stricture of a poset on any subset of the strata involved. Any 
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union of lower intervals in this poset defines a closed subset in the above 
topological space and any sequence of embedded systems of lower intervals 
defines a filtration by closed subspaces. 


8.9. Proof of Theorem E. Let us first cover the nice case. We consider 
the Leray spectral sequence for the cohomology with compact supports as- 
sociated with the standard filtration of a nice C),4Cz,g. By definition, we 
consider the filtration of Ci,. %Cz,g by closed subsets F? such that F* is 
formed by the union of all strata of (complex) dimension less or equal to 7. 
The (p, q)th entry of the first page contains the (p+ q)th cohomology with 
compact supports of F?*+1 mod F?, which coincides with the direct sum 
of the (p + q)th cohomology with compact supports of all refined double 
strata of dimension p. Let (C*)* x (C)! be one of the strata of minimal 
dimension in the considered nice Cj,q %Cz,g. It follows from Proposition 
9.1 below that the (p+ q)th term is the direct sum of several copies of the 
(p + q)th cohomology of some (Ci eee c'-@-1)_ The differential 
d, coincides with the connecting homomorphism in the exact sequence of 
triples HP+9(F?+1, FP?) + Hptatl(FP+?, FP+?), The crucial property of 
differentials in this spectral sequence for the filtration by quasiprojective 
subvarieties is that they respect the Hodge weights (filtrations) in the coho- 
mology, see [D12,D13]. By Remark 8.1, the Hodge weights of all elements of 
the (p,q)th term are equal to p— (k+1),p—(k+1). Thus, the only nontrivial 
differential in this spectral sequence can be dj, since only d,; respects the 
weights. 

Now, let us modify this proof for the almost nice case. As it was men- 
tioned before, the filtration of an almost nice Cy, MCo,6 by dimensions is 
a filtration by closed subsets, and rows of the corresponding spectral se- 
quence contain the cohomology with compact supports of unions of strata 
of a given dimension. The union of all strata of a given dimension in the al- 
most nice case is a quasiprojective variety with the standard filtration. The 
corresponding Leray spectral sequence, obviously, degenerates at E1, since 
by Proposition 9.1 below all refined double strata of the same dimension 
have the same form and none of the differentials in this additional spectral 
sequence respects the Hodge weights. Thus, one gets the same cohomology 
with compact supports for the union of these strata as for their disjoint 
union. Therefore, exactly the same degeneracy arguments as above apply. 


8.10. Examples. The authors have calculated Betti numbers for all in- 
tersections of top-dimensional Schubert cells in F3 and F4, which are all 
nice, using a different method. (In dimension 5 the intersection of the op- 
posite top-dimensional Schubert cells is only almost nice, see Fig.9a.) The 
only interesting cases in these dimensions are g = 321, 0 = 4231, 3421, 4312 
or 4321. These Betti numbers show that in all these cases the differential 
d, acts in the maximal way, i.e. that all rows of the first page form acyclic 
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complexes except for the top dimension, see an example in Fig.10. The 
second page in all these examples contains only one nontrivial column, and 
the cohomology in this column has pure weights; thus, the mixed Hodge 
structure is pure. 

Let us consider the filtration F? > F2 > F), where F! = A, F? = 
AUB, P= C1234,4231 MN C4231 4231, and use the Leray sequence, see Fig.11 
and Example 7.2. This filtration is precisely the one obtained from the 
adjacency partial order. From the geometrical description of the strata A, 
B, C one can see that the nontrivial differentia] d, acting from the first 
column to the second one (pay attention to the weights) kills the whole 
first column, thus leaving us with the second page, which again contains 
the pure Hodge structure. 


8.11. Counterexample to the purity of the mixed Hodge structure 
of Ci.49C,,g. In all previous examples the so-called purity of the mixed 
Hodge structure holds, which in the case of Ci,2C,,g means that the only 
nonvanishing Hodge numbers are he and they are equal to De = (—1)*x2". 
However, this turned out to be false in general (for arbitrary Cia Gas) 
for the following reason. If purity holds for some Cia Cog, then the 
coefficients of E%*(z) must have strictly alternating signs. Recently, using 
computer, B. Boe has found 4 intersections in Ge whose E-polynomials do 
not have strictly alternating signs. Two simplest of these examples have 


Ist page 2nd page 


Figure 10. Leray spectral sequence for the nice intersection 
a = 6 =o = 4321. Arrows show nontrivial differentials, 
pairs of numbers in the left upper corners are Hodge weights 
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Ist page 2nd page 
Figure 11. Leray spectral sequence for the simplest nonnice 
intersection a = 8 = o = 4231; arrows show nontrivial differentials 


been checked by the authors using the above method, and here is their 
description. 

Ap) C'123456,563412 NC654312,654321 consists of the following 24 refined dou- 
ble strata: 1 copy of (C*)® x C5, 6 copies of (C*)> x C*, 10 copies of 
(C*)” x C3, 6 copies of (C*)® x C?, and 1 copy of (C*)'! x C. The corre- 
sponding E-polynomial is equal to q(-1 + 5q —- 11g? + 1393 — 7q* — q? + 
g° +7q7 — 1398 + 11q° — 5g? + q!), and its coefficients do not have strictly 
alternating signs. 

2) C123456,563412  C654321,654321 consists of the following 41 refined dou- 
ble strata: 1 copy of (C*)? x C®, 5 copies of (C*)* x C*, 12 copies of 
(C*)® x C3, 15 copies of (C*)® x C?, 7 copies of (C*)!° x C, and 1 copy of 
(C*)!?. This E-polynomial is equal to 1—5q +119? — 139° + 8q4 — q — q° — 
q’ + 8q8 — 13q9 + 11q1° — 5g"? + gi2. Considerations of the adjacency par- 
tial order reveal that both of these examples are almost nice, but not nice. 
Since by Theorem E the corresponding Leray spectral sequence degenerates 
at E, it will be interesting to calculate d,, and therefore the cohomology 


of these intersections. 


9. Combinatorial properties and applications of E-polynomials 


9.1. Proposition. For any nonempty stratum, the sum of the (complex) 
dimension of the torus and the doubled dimension of the linear space equals 
Ing(a) +1ng() —Ing(c), where Ing is the inversion length of a permutation. 
In particular, all strata of the same dimension have the same form. 


Example. For the intersection C1234,4231 C'4231,4231 we get Ing(a) = 
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Ing(@) = Ing(c) = 5, and I(St) + 2d(St) = 5 for all strata. 


9.2. In order to prove Proposition 9.1, we start with the following lemma 
relating the length of a permutation o to the length of a, (see Lemma 3.4). 


Lemma. Let s = (i1,...,i,), then 
Ing(a;) — Ing(c) = I(s) + 2d(s) — 7, — a(t) + 2, 


where I(s) and d(s) are the reduced length and the domination of s, respec- 
tively. 


Proof. Indeed, let us denote by I; the set of consequtive elements of o 
lying between i; and 13-1 (we set ig = 0). When passing from a to Os, We 
loose all the inversions containing 21, and, for each i;, 2 <j < k, all the 
inversions (7,7;) such that 7 € I; and i > a;. On the other hand, for each 
453, 2 <j <k, we acquire new inversions of the form (i;,7) such that i € ils 
and 7 <1,;. So, 


k 
Ing(as) — Ing(c) = SoH Cli <1) — 


j=2 


k 
DH Ty: i> ij} Hie hii> i} Aid hii<ij}a 


j=2 


k k 
Dott ely: <ij}- #1 > ij} — Blas < ay}. 
pl 


Fi 


Evidently, the first term in the right hand side equals d(s), while the 
last term equals i, — 1. Next, the sum of the first and the second terms is 
ye #{i: i € I;}, and hence it equals n—k— (cae) — cs (pia 
Taking into account that I(s) = k—1 (see 0.8), we get the statement of the 
lemma. & 


Example. Let o = 46237518 and the decreasing subsequence in it be 
(6,3,1), then o, = 4321657. We have Ing(7) = 12, Ing(o,) = 7. On 
the other hand, d(s) = 2, I(s) = 2, i, = Oe (Os) = % Sa, ¢= 12 = 
4+2-—6-742. 

9.3. Proof of Proposition 9.1. The proof is done by induction. In the 
cases n = 1 and n = 2 the formula is trivially true. 

Step of induction. For any nonempty stratum St € Cy, NC, we want 
to prove that /(St) + 2d(St) = Ing(a) + Ing(@) — Ing(c), where (St) is the 
total length of St and d(St) is its total domination (see 0.11). Assume that 
the chain corresponding to St starts with the permutation o, and that the 
next permutation of the chain is o,. By the inductive hypothesis, we have 
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1(St’) + 2d(St’) = Ing(a’) +1ng(#’) —Ing(o,), where St’ is the refined double 
stratum in Cy,9 Cg, ,g corresponding to the subchain of the initial chain 
starting with o,, a’ and (’ are obtained from a and #, respectively, by the 
reduction of the first element, see Proposition 0.16. Subtracting the above 
two formulas and applying Theorem B, we arrive at 


I(s) + 2d(s) = Ing(a) — Ing(a’) + Ing() — Ing(6’) — Ing(o) + Ing(as). 


Obviously, Ing(a) — Ing(a’) = a(1) — 1 and Ing(@) — Ing(6’) = A(1) — 1. 
Hence, by Lemma 9.2, we have to prove that a(1) + @(1) = 11+ Omran) 
However, by Theorem C, i; = (1) and o~*(ix) = G(1), and the result 
follows. a 


9.4. Remark. The assertion of Proposition 9.1 remains valid for inter- 
sections of Schubert cells in all flag spaces G/B. 


9.5. Lemma. The intersection Cia Co,wo 18 isomorphic to ChaM 
CG eusewa Oley. LeU, Cra ACo,w, 1s nonempty if and only if Cia 

wo,woow 18 nonempty, that is, in the case a = woo, where = denotes the 
usual Bruhat order. If Ci,a.Co,wy is nonempty, then tts dimension equals 


Ing(a). 
Proof. See [KL2] 1.4, p.187. k 


9.6. Corollary. Let s; sq be two decreasing subsequences with coincid- 
ing first and last elements, then 05, X Os, and Wo0s, = W0%s2- Thus, if 
Cra! VCo,, wo 18 nonempty, then Ca NVCo,, wo 28 also nonempty. 


Proof. The set of all decreasing subsequences in o with fixed first and last 
elements describes the set of all G,-orbits that have a fixed set of dimensions 
of intersections with the subspaces of f and g, see Corollary 5.5. Moreover, 
by Lemma 6.2, the partial order + describes their adjacency. Thus, any 1- 
parameter family of lines I(t), t € [0, 1], such that I(t) € Oc,s, fort € [0, 1) 
and I(1) € Og,s, satisfies the assumptions of Lemma 6.3 and gives the 
necessary assertion. a 


Example. For o = 83427561 and s1 = (8,3,2,1), s2 = (8,7,5,1) we get 
Os, = 3241756 F og, = 7342516. 

9.7. We now start proving Proposition 0.27 in the case B = wo (or a = wo, 
which is the same), i.e. for intersections of pairs of Schubert cells one of 
which has the top dimension. This class of pairwise intersections is closed 
under the projections involved in our algorithm, since the subset of flags 
in a top-dimensional cell containing some fixed line is isomorphic to the 
top-dimensional cell in the quotient space. 
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We proceed by induction on n. Cases n = 1 or 2 are obvious. 

Step of induction. Assume that the assertion holds for the refined 
decomposition of F,,_; and consider the refined double decomposition of 
ChaCow. in F,. By Theorem C, refined double strata in Cie 1) Com, 
are diffeomorphic to Cartesian products of orbits Os (where s € S¥°) by 
refined double strata in Cia ml Co,,w,, where w4 is the longest permutation 
on 7 — | elements and a’ has the same sense as before. It suffices to 
show that nonempty refined top-dimensional strata corresponding to all 
s € S¥° have no gaps in dimensions. The dimension of any nonempty 
intersection Chal Co,45,wo, and thus the dimension of the top-dimensional 
strata, equals Ing(a’) independently of s, see Lemma 9.5. Thus, we have 
to show that the numbers Ing(a’) + dim(O,) have no gaps when s runs 
over the set of all decreasing subsequences in S*”° such that Cra Coa 
are nonempty. Since Ing(a’) does not depend on s, we must show that 
the dimensions of all orbits corresponding to the decreasing subsequences 
s such that a’ > woo,have no gaps, cp. Lemma 9.5. The latter statement 
follows from Lemmas 6.2, 6.3 and Corollary 9.6. a 


9.8. Now we cover the second case of Proposition 0.27, ¢ = wo. 

By Lemma 9.5, the intersection Cra (Gees isomorphic to Chal 
Case Cow. Thus, the absence of gaps in dimensions of refined dou- 
ble strata in the case o = wo will follow from the preceding considerations 
in the case B = wo and the following 


Lemma. There exists a bijection between the refined double strata in Creal 
Co,wo and those in (Come (Op Cyan, oS Ci cw. Strata in Ori (8) Cree 
diffeomorphic to Cartesian products of the corresponding strata in Cian 
Can anaeen by Clone . 
The proof is based on the following observation. 

9.9. Proposition. Given the polynomial E%*(z) of some intersection 
Cia Co, and two numbers k and | such that the top-dimensional strata 
in the refined double decomposition of CraNCo,g have the form (ey x(Cie 
one restores the number of strata in Cyaan) Cy,8 in all dimensions. 


Proof. Since the dimension of Cia Co,¢ is k +1, the degree of E28 (z) 
is also k + 1, and the leading coefficient @%41 is equal to the number of 
top-dimensional strata. Consider Ble) a441(z—1)*z! and assume that 
its degree is p < k +1. Then its leading coefficient Gp gives the number 
of strata of dimension p in Ci,aMCyz,g. By Proposition 9.1, the strata 
of dimension p have the form (Cr)? ala (C) 2a Subtracting from 
ESF (z) = aps i(z — 1)*z' the polynomial ap(z — Nt one gets 
the number of strata in the next dimension, and so on. &) 


9.10. Proof of Lemma 9.8, By Lemma 9.5 and the definition of E28 (z) 


Pairwise intersections of Schubert cells 431 


one has Bwoworwo(z) = z!ne(owo) R7~0(z). Moreover, it is known that both 
Cr aNCup,woows and Cy,a¢ACow. are irreducible varieties, and one can show 
that their only top-dimensional strata have the form (C*)!"8(#)—!»s(owo) and 
(C*)!ng(a)—Ing(owo) x Clns(owo), respectively. (These strata are obtained by 
using maximal decreasing subsequences on all steps of the algorithm.) Now, 
applying the procedure of defining the number of strata of each dimension 
described in Proposition 9.9 and using the result for the first special case, 
one gets the necessary statement. S 


9.11. Here we recall the relation between the values of structure constants 
C2) ie Hecke algebra and the E-polynomials, cp. [KL1, Ka, Cul]. 
This relation holds for any flag space G/B where G is a semisimple group. 

The Hecke algebra H is the deformation of the group algebra of Gp. 
As a linear space H has the standard basis {T,,}, w € Gn. As an algebra 
H is spanned by {T;,}, where t; is the 7th simple transposition, satisfying 
the Hecke algebra relations. Namely, 


liye Tiss #1, = Tesys * Ty, * ay 
Te * Ti, =T,, alee 
dy, ae => (z = NE. + z1y. 


The structure constants cw ,,,, are the coefficients in the decomposition 
? 
wy * Le = ae Cw} we W3° 


Proposition. Ec, .nCo,s(2) = ©2,-1 (2): 


Proof. If z is a power of prime, then the structure constant ¢,% 4, (z) 
calculates the number of points in the intersection Cy ral) OF sin see 
e.g. [Cul]. In an appropriate basis of C”, all refined double strata in Fp, 
are defined over Z, and thus the refined double decomposition is defined 
simultaneously over any finite field GF. Any refined double stratum over 
GF is isomorphic to an appropriate (GF*)* x (GF)! where GF" is the mul- 
tiplicative group of GF. Both Ec, .nc,, a(z) and c? g-1(2) are polynomials 
in z with integer coefficients coinciding when z is a power of a prime, since 
they both count the number of points in Ci,2Co,g over the corresponding 
finite field. Thus, Ec, .ACz,¢(2) = c% g-1 (2): i 


10. Relation to the decomposition of Curtis—Deodhar 


10.1. In order to present to the readers certain similar results due to 
V.V.Deodhar and Ch. Curtis, we quote several statements of [Del] con- 
structing a special decomposition of Ci,a and C),¢ 1 Cwo,6 in terms of 
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reduced expressions for a~1. We preserve mostly the original notation. Let 
W(y) denote the lower interval of y in W in the usual Bruhat order. 


10.2. Proposition ([De1], Theorem 1.1). The Bruhat cell By -B can be 
decomposed into disjoint nonempty subsets {Dz}cep (D is the indexing set, 
which can be described explicitly) such that 

(i) By-B= Usep Dg. 

(ii) For each o € D there exist unique non-negative integers m(a) and 
n(g) such that Dg ~ C™2) x (C*)"@), 

(iii) For each o € D there evists 1 © W (recall that in our case W = Gy, 
and so Dg C B~z-B with B~- = woBwo). This element x is unique, 
belongs to W(y) and is denoted by ™(a). Thus one gets a map 
nm: D> Wy). 


10.3. Corollary ([Del], Corollary 1.2). By-BNB-2z-B = ee D(a) ee 
(In particular, By - BO B-«-B # © if and only ifxeE W(y).) 
10.4. Let U* and U- denote the maximal unipotent subgroups in SL,, 
corresponding to the set of all positive and negative roots, respectively. 
(In our case they are the usual upper- and lower triangular unipotent sub- 
groups.) Fixing y € W and its reduced expression y = s)...8,%, where 
8; =t;;, let us define for all 1 < j < k the subsets U; =U* 1\*F"*®U— and 
UJ =U+m**sU*_ (For a subset A € SL, andgéSL,,9A= gAg=*): 
Proposition ({De1], Lemma 2.2). 
(On Sie Ui Sy 8 Sy 2) gang = icie 
(i) UwEU? 6.6 Ure yet 
(ili) For any j,1<j <k+1, one has U+ = U; -*3"**U4 and such an 
expression is unique. 


(iv) Any element € € By -B can be uniquely written as US1...8, with 
uEeUuy,. 


10.5. The indexing set D in Proposition 10.2 is described in terms of a 
reduced expression y = s.. -Sk, Where k is the length of y. It consists 


of subexpressions 51... 5p, ---Spm---8k With some additional properties, 
which are called distinguished. More precisely, one can regard a subexpres- 
sion as a sequence g = (gp,..., ox) of elements of W such that (i) o9 = id 


and (ii) O50; € {id)s, ioral <r Norethat tor any j one gets 
that o;_; and o; are always comparable in the Bruhat order, i.e. one has a 
trichotomy o;_1 < 0;, or Oj—-1 = 93, OF oj_1 > oj. Let us denote by n(a) 
the number of all positions j such that 03-1 = 0; and by m(c) the number 
of positions 7 such that oj-1 > 0;. Finally, let m(c) = oy. 

Now we can define the index set D in Proposition 10.2. A subexpression 
g is called distinguished if it satisfies the additional condition (iii) oj < 
o;-18; foralll <j <k. 


Proposition ([De1], Proposition 3.1) Let u; € U, be fixed. For 0 ye 
let a; € W be the unique element such that U1$1...5; € B-a;B. Then 
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go = (00,01,---,;0K) is a distinguished subexpression. 
In this way one gets a map v : U; — D defined in an obvious way. 


10.6. Proof of Proposition 0.28. Let us define the standard reduced 
expression of y € G,, by the following rule: y = ty-1(1)-aty-1(1)-2 °° HY, 
where y belongs to Gn_1, ie. g(1) = 1, and ¢; is defined in 3.1. We 
consider 7 as an element of G,_, acting on the set {2,3,...,n} and define 
the standard reduced expression inductively. 
Example. 3241 = (4,3)(3, 2)(2, 1) - (1324) = (4, 3)(3, 2)(2, 1)(32). 

Let y = t,...t, be the standard reduced expression and q = y~‘(1). 
Let us consider the result of Deodhar’s decomposition of By - B after the 
first g — 1 steps, which corresponds to the sequence Uj; D> *'U2 D ... 
>$1-Sa-2 U_ (see Proposition 10.4 above). We denote by v; the spanning 
vector of the one-dimensional space of the flag f € By- B. It has the 
following properties: v? = 1, vj = 0 if j > q. The first g —1 steps of 
Deodhar’s procedure decompose the cell into the disjoint union of 27-1 sets 
enumerated by all possible subsets of nonvanishing coordinates of vy. At the 
same time, our main algorithm for Cy,q (with a = y+) produces the same 
result after the first step. Moreover, since the elementary transpositiopns 
Sq,--- 8 do not contain the element 1, one may assume that the vector v, 
is fixed on all the subsequent steps. An easy reformulation of Deodhar’s 
procedure presents the subsequent steps of his algorithm as an application 
of the previous steps to the quotient space and quotient flags mod v;. Thus, 
it gives exactly the same result as our main algorithm. ia 


11. Final remarks 


Below we formulate several combinatorial and topological problems re- 
lated to the material of the preceding sections with some comments. 
11.1. Combinatorial questions. Problem 1. Calculate the number 
of strata in the flag completion of Grays 

One can use the very weak Bruhat order on Gp defined in 81. By 
Proposition 1.5, the number of flags in the flag completion of ( f,g) in 
relative position o equals the number of elements in the lower interval of 
o in this order. Proposition 1.7 provides an inductive way to find this 


number. 

Problem 2. Calculate the number of strata in the refined double decompo- 
sition RD;,, of the whole flag space F,, (see Fig. 1 and the description of 
the main algorithm). 


As a preliminary question, one needs to find the number of decreasing 
subsequences in a permutation. Fig. 12 contains the number of decreasing 
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subsequences and refined double strata for all permutations in G4. 

The next group of questions deals with the properties of the adjacency 
partial order. Let us introduce the following natural linear lexicographic 
order on permutations and chains of permutations. Namely, a permutation 
@ is said to be bigger than 3 if the leftmost element of a that is distinct 
from the corresponding element of f is bigger than the latter one. This 
lexicographic order can be extended naturally to chains of permutations. 
(The chains on all figures are arranged from left to right according to this 
lexicographic linear order.) 


Conjecture 1. For any nonempty Ci.aMCo 4, the refined double stratum 
that is the maximal element in the lexicographic linear order is also the 
mazimal element in the adjacency partial order. Moreover, it is one of the 
top-dimensional strata. 


Conjecture 2. For any nonempty C1,aCo a, the refined double stratum 
that ts the minimal element in the lexicographic linear order is one of the 
strata of the minimal dimension. 


permutation number of flags number of number of refined 
in flag completion decreasing double strata 
subsequences 
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ts 
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Figure 12. Some combinatorial characteristics for permutations in G, 
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Conjecture 3. Let S and T be two refined double strata such that S = T 
(in the adjacency partial order), and assume that dim S—dimT > 1. Then 
there exists a sequence of intermediate strata S = S, = Sg = S3m--° = 
S; =T such that dim S; — dim S;_; = 1 for alli. 


Consider the sequence of numbers of strata (contained in some C1,4M 
C,,g) ordered by dimensions. For example, these sequences for 2 counterex- 
amples of B. Boe are 1,6,10,6,1 and 1,5, 12, 15, 7, 1. 


Conjecture 4. The above sequence is unimodal, that is, nonstrictly in- 
creasing up to some place and nonstrictly decreasing after that. 


B.Boe has verified this conjecture for G4 and G5, as well as for the 
Coxeter groups By, C4, and Ds. 


11.2. Topological questions. Problem 3. Describe combinatorially 
the closure pattern of refined double strata, that is, enumerate all strata St; 
such that StNSt; # S. 


Problem 4.Give a combinatorial description of the differential d, in the 
Leray spectral sequence of §8 for (almost) nice Ci,a 1 Co,p- 


Consider the standard filtration of any (e.g., hard) C1,4%Co,g by closed 
subsets, see 0.25. 


Conjecture 5. The associated Leray spectral sequence degenerates at E4. 


Another interesting topic concerning intersections of Schubert cells is 
the relation between their topological properties over R and C. We say 
that a real algebraic variety V enjoys the M-property if the sum of its Betti 
numbers with coefficients in Z/2Z coincides with that of its complexifica- 
tion. It is shown in [Sh] that the intersection RCY wo ARCup,wo enjoys such 
a property. 


Conjecture 6. Any intersection RC1,09 RCo,g enjoys the M-property. 
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